XClickText misses vertical offset/ get networkerror

gup.mi · April 30, 2025, 1:39pm

Hello everyone,

I have created a basic macro to select the vehicle category on a car rental website.
However, I am facing two issues:

When running the macro on my PC the OCR seems to identify the correct category, but the XClickText command clicks the row below.
When executing the macro on a Windows Server 2019 machine, I encounter an error (NetworkError: Failed to execute ‘importScripts’ on ‘WorkerGlobalScope’: The script at ‘chrome-extension://gcbalfbdmfieckjlnblleoemohcganoc/lib/tesseract/core/tesseract-core-lstm.wasm.js’ failed to load.)

Below the macro (some commands are disabled with comment).

{
  "Name": "Upr99-ScSegmFORUM",
  "CreationDate": "2025-4-30",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.unipolrental.it/offerte/aziende-e-partite-iva",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "store",
      "Target": "${!cmd_var1}",
      "Value": "Segm",
      "Description": ""
    },
    {
      "Command": "store",
      "Target": "${!cmd_var2}",
      "Value": "Param",
      "Description": ""
    },
    {
      "Command": "store",
      "Target": "Monovolume",
      "Value": "Segm",
      "Description": ""
    },
    {
      "Command": "if",
      "Target": "${Param}==\"CANC\"",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=//div[13]/button[2]/span[2]",
      "Value": "",
      "Targets": [
        "xpath=//*[@id=\"__next\"]/div/div/div[3]/div/div[13]/button[2]/span[2]",
        "xpath=//div[13]/button[2]/span[2]",
        "css=#__next > div > div > div.undefined > div:nth-child(1) > div.ProdottiListing_selectedFilters__5uAu3 > button.Button_button__rfKht.Button_button--ghost__liNbO.Button_button--md__hlsv8.undefined.Button_button--icon-left__7jHyC > span.Button_label__aY_y3"
      ],
      "Description": ""
    },
    {
      "Command": "end",
      "Target": "",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "",
      "Target": "",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=//div[12]/div/button",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "visionLimitSearchArea",
      "Target": "area=150,150,700,700",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XClickText",
      "Target": "${Segm}",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=//*[@id=\"__next\"]/div/div/div[3]/div/div[12]/div/button/span",
      "Value": "",
      "Description": ""
    }
  ]
}

Thanks for your help

noahhath · April 30, 2025, 6:03pm

I have no idea why everyone is using xclick. That is the most unreliable way to automate a website. The best way it to use xpath to detect buttons. (I know there is a little learning curve, it you never heard of it)

wrote a quick script using xpath instead of xclick and it worked for me:

{
  "Name": "test 1",
  "CreationDate": "2025-4-30",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.unipolrental.it/offerte/aziende-e-partite-iva",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=(//span[text()='Categoria veicolo'])[4]",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=(//span[text()='City Car (3)'])",
      "Value": "",
      "Description": ""
    }
  ]
}

gup.mi · May 2, 2025, 8:44am

Hello Hendrik, thank you very much for your assistance.

Today, the number of “City Cars” changed from 3 to 4, requiring me to modify the macro. Since this number varies daily, is it possible to use wildcards?

To address your question:

Uivision emphasize its distinctive “visual” approach, as it is sometimes the only method that works, which is why I requested a solution to the issue.
The Xpath recorded by Uivision differs in syntax compared to yours. For example: xpath=//*[@id=“__next”]/div/div/div[3]/div/div[12]/div/div/button[3]/span/svg
instead of your xpath=(//span[text()=‘City Car (4)’]),
which is far easier to read.

I would like to learn more about this. Do you have suggestions? Thank you.

gup.mi · May 7, 2025, 10:09am

Wildcards problem solved using xpath=//span[contains(text(),‘City Car’)]
Still interested in learning about xpath language

admin · May 7, 2025, 10:29am

I totally agree with @noahhath that if the xpath approach works, it runs faster than visual automation - and we recommend it as well

Having said that, I looked into why XClickText did not work on this dropdown and here is what I found. There are actually two issues:

(1) Dropdown menus are technically (often) like select box dropdowns and are floating above the website, so they are not included in website screenshots! This is why in browser visioon mode, Ui.Vision does not see this menu! This is what you get when you click the FIND button:

Solution for #1: Use desktop mode to “see” select menus.

(2) The second issue is that “City” and “car” are in two different bounding boxes (the gray box in the background of the red word in the overlay". That is a limitation of the OCR layout logic. This is why there is no match for the word “City car”.

Solution for #2: Use only the first word or second word to match it: XClickText | city

With 1+2 solved, XClickText works fine:

Macro:

{
  "Name": "ocrtext",
  "CreationDate": "2025-5-7",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.unipolrental.it/offerte/aziende-e-partite-iva",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "store",
      "Target": "98",
      "Value": "!ocrengine",
      "Description": ""
    },
    {
      "Command": "click",
      "Target": "xpath=//*[@id=\"__next\"]/div/div/div[3]/div/div[12]/div/button/span",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XDesktopAutomation",
      "Target": "true",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XClickText",
      "Target": "city",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XDesktopAutomation",
      "Target": "false",
      "Value": "",
      "Description": ""
    }
  ]
}

noahhath · May 7, 2025, 3:39pm

I’m usually learning best (xpath) by looking into examples and adjust them to my need.
I great help is chatGTP for that

admin · May 8, 2025, 8:47am

→ For the second issue I made a new forum post to keep the topics separated, as they are not related.