Area Coordinates for visionLimitSearchArea

Ui.Vision General Discussion
1

Is there a “Select” functionality to choose an area of the screen for the ‘area= x1,y1,x2,y2’ target? (like you can press “Select” next to the Target field to choose an element on a website by clicking)

Otherwise, how do I know what my coordinate system is on my display? Further, how do I specify this across two monitors?

I know what region of a screen my intended visual search is by looking at it, but not how to get the numeric coordinates.

Thanks.

2

Is this for web automation (inside browser) or for desktop automation?

3

Tools to get the x/y coordinates:

visionLimitSearchAreaRelative:

Another option: Instead of visionLimitSearchArea I often use visionLimitSearchAreaRelative to visually mark the image search area. It is used in DemoXDesktopAutomation, too. And in the RPA select box automation demo :

image

4

Thanks for sharing these threads!

It’s for desktop automation. Technically Chrome’s built-in PDF viewer, but UI Vision doesn’t appear to be able to interact with that tab like a normal browser tab.

5

The links above gave me a method of getting the X/Y coordinates (Mac using Cmd-Shift-4).

Now the question is, how do I specify which display to reference for the coordinates?

I have my Macbook Pro built-in display, plus an external display. Do these visual commands have the ability to handle multiple monitors?

6

PDF automation in Chrome can be done in browser mode! Then the browser x/y coordinates apply. You are right that the Selenium IDE-style web commands fail on the built-in PDF viewer, but the visual XClick and XType work fine! Example: The DemoPDFTest_with_OCR script that ships with UI vision includes PDF automation.

Multiple monitors do not disturb the automation (they can stay connected) but the automation itself needs to run on the main screen.

But since XClick uses the global mouse cursor, you can not do any other work on your machine while the automation runs anyway. We often run the automation in some small Linux virtual machine, which can then run in the background.