Identify multiple circular areas and perform actions one by one

Ui.Vision
1

On a webpage, there is a image area implemented through a canvas.
In the image, there are many circles with only two colors, and their sizes are not fixed (determined by different data).

image
image1920×928 40.4 KB

When the mouse moves over a certain circle, a popup will appear.

image
image1920×893 49.5 KB

CASE: The RPA action I want to implement is to identify all the graphical areas, then move the mouse over one of the circles, display the popup, and identify its content. Then, loop and move the mouse over another circle.

Q1. The content I identified using visualSearch is very inaccurate, and I can’t find all the circles.
Q2. How should I implement this case?

1 Like
2

Wow, that is a very interesting but very tricky task :grinning:

identify all the graphical areas

This is the tricky part here, especially if you really need to be sure to have all red/green points and not to miss some. Once you have the x/y moving the mouse and then doing OCR on the image is rather easy… but reliably finding all the points… hmmm… …I am not sure.