How to use XClickText and XClickTextRelative?

UI.Vision RPA General Discussion
1

Hi all,

I’m trying to replace some code that works fine with XClick/XMove in one machine but fails in other due to the screen resolution.

In this thread they suggest to these two OCR commands as a workaround, but I’m not clear in how to select the correct position since if it is resolution independent, is not by pixels or X/Y coordinates.

May someone explain how is the process to know which value move to left/right, up/down?

Additionally, I’ve been trying to make some simple tests with this site but the string is not found.

Only 2 steps:
1-) Locate “example X0C2” with XClickText to write something there
2-) Locate “Log in Another Way” with XClickTextRelative in order to understand how much move to up and right to click “Find button”.

I 've seen command descriptions and video explanation but I’m still have doubts. Thanks

XClickTextRelative
XClickTextRelative995×556 24.7 KB

This is my 4 commands macro that is stopping in 2nd command since text is not found.

Line 2: OCR for ‘example X0C2’ not found

{
  "Name": "test2",
  "CreationDate": "2024-5-22",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.examlogin.com/#/login",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XClickText",
      "Target": "example X0C2",
      "Value": "",
      "Description": "Locate Code box"
    },
    {
      "Command": "XClick",
      "Target": "ABCD",
      "Value": "",
      "Description": "Write ABCD"
    },
    {
      "Command": "XClickTextRelative",
      "Target": "Log In Another Way#R20,5",
      "Value": "",
      "Description": "To click in Find button"
    }
  ]
}
1 Like
2

This is a good question, and a very good timing. In the last weeks we have been working on improving the XClickText and XClickTextRelative commands. The next RPA update brings more local OCR options (even on Linux), more OCR languages and wildcard (* and ?) support for more control over text matches.

The following demo already uses the new version:

The video also shows that text recognition works better than image recognition if the screen resolution is dramatically different between system.

macro that is stopping in 2nd command since text is not found.

I found this issue. In the macro you use: "Target": "example X0C2",

This is formally correct but in order to make text recognition commands more robust, we recommend to use the least number of words that uniquely identify a position. In this example, just using the word “example” will do.

The found mouse click position is stored in the ${!ocrX} and ${!ocrY} variables. This works the same as the ${!imageX} and ${!imageY} variables for image recognition. (This is not needed here, I am just mentioning it for completeness).

I will update this post again once the new version is released. It should not take long :wink:

3

Hi, thanks you for your answer and video. I still have the doubt how did you know that in 4th command the value to search “Find” button should be 35,4. How do you identify that? Test and error or how?

I understand about to use minimum number of words to make OCR search precise, but when I was trying to select a practice site for you to show, I tried to search a word that appears more than once in the page. Then I needed to search by 2 works separated by space and the result was found in another side of the page even the 2 strings were not present.

Thanks for you help again

4

Yes, test and error. I start with a good guess (or simply e. g. 10,20), press find and see where it “hits”. Then I adjust the values as needed, and press “Find” again. Usually, after 3-4 rounds I have a good value.

It is important to note that these values (10,20 etc) are not x/y values, they are multiples of the “average length of a character” that Ui.Vision calculates internally. The important part is this value stays constant when the screen scaling changes. So once you found good parameters for XClickTextRelative on one machine, the command work on every machine without changes.

The same applies to all other …TextRelative commands:

  • XClickTextRelative,
  • XMoveTextRelative and
  • OCRExtractTextRelative.
1 Like
5

Thanks for your explanation. I’ll delete my other question related with this.

Regards

6

V9.2.0 is available now :slight_smile: