UI.Vision RPA - VisualAssert check colors

Hello,

I have an elements wich background colors are changing (drawing will remain the same), so I am tring to test it with visualAssert, but I always get a score of 1 (or roughly 0.9x) even if the colors is not the good one (eg. red instead of green, …).

Is it normal that VisualAssert doesn’t check color ?
Is there a way to correct it ?

Thank’s

Can you extrapolate from the page code (html, css) the RGB code it is using? You could get it as a second step check.
About the visualAssert, I didn’t find a reference about the way it treats colours in the documentation, so I can really help on that, sorry

For exampleI got an expected button with red background rgb (220, 53, 69) but having green one (40, 167, 69), visual comparaison

I manage to get the information (and to control it) by getting DOM element but I’m expecting to check only by visual comparison as totaly black box / not doing hypothesis on DOM structuratoin

Hi @ECuinet - you are correct that the current computer vision algorithm puts emphasis on form over color, but I understand that sometimes the color is key.

=> Can you please post some screenshots of the buttons? Maybe we have an idea for a workaround. For example, is it a solid color?

An example, so yes it’s solid color which could be different depending of conditions …

2020-05-26 164337

Is the position relatively stable on screen? You could delimit search area, and make sure it will search for that button, only.

@Jorgelhus suggestion with visionLimitSearchArea is good if the goal is “just” to find a certain button.

So to clarify, what exactly is the goal:

  • Reliably identify and find e. g. a green, blue or red button? This means you have 3 different UI test input images and want to find the right button on the screen for each input image.

  • Or find the button (regardless of its background color) and then find out what background color it has?

It’s the first option, compare part of web pages with different UI test pictures from specifiation (the only modification is background of a button).
Will try with visionLimitSearchArea tomorrow because position is stable

I have tried with visionLimitSearchArea, same result (which is logical if vision algorithm emphase forms and not color).

But I got some strange comportment during the following test :

  1. Various step (in my case set proxy to simulate network failure which will change button from green : (class badge-success) to red (class badge-danger)
  2. Check button state with assertElementPresent //*[@id=“indicateurConnexion”][contains(@class,“badge-danger”)]
  3. Limit searchArea to the button
  4. visualAssert to check the color of the button (image = button included background + text)

Step 2 passed with success, but after step 4 the _lascreenshot.png I got in Screenshots tab is displaying button with green background, so screenshot is not representating current step (timestamp of screenshot is ok and compliant with time of exuction of step 4).

JSON corresponding to step 2 to 4 :

    {
  "Command": "assertElementPresent",
  "Target": "//*[@id=\"indicateurConnexion\"][contains(@class,\"badge-danger\")]",
  "Value": ""
},
{
  "Command": "visionLimitSearchArea",
  "Target": "element:xpath=//*[@id=\"indicateurConnexion\"]",
  "Value": ""
},
{
  "Command": "visualAssert",
  "Target": "EnLigne_dpi_96.png",
  "Value": ""
},
  1. I confirmed this _lastscreenshot.png not updated issue (see here: 05.28.2020-16.10.48 )

  2. VisualAssert is almost colorblind, as discussed above.

Solution: We will add a new getPixelColor (x,y) command soon. (Available now) This way you will be able to check on the color:

  • visualVerify | (image of button)
  • visualGetPixelColor | ${!imageX}, ${!imageY} | var_with_color
  • if | ${var_with_color} == "#5da225" | …then do something

Screenshot VisualAssert: All the button edges look almost the same to it´:

Update 2020/8/8: visualGetPixelColor is now available :slight_smile:

Related forum posts:

visualGetPixelColor will be a good solution for my problematics, will wait until implementation.

Thank’s

This issue is fixed with V5.7.5.

Note that visualGetPixelColor is not yet in this “point” update, but it will be part of the next larger update.

visualGetPixelColor is available in V5.8.8