UI.Vision 10 Beta — AI macro creation and new real browser clicks that need no focus

Announcements
1

Early beta, soft launch. Only announced here :grinning_face:

Install link: This beta installs as separate Ui.Vision extension.

:backhand_index_pointing_right:Your existing UI.Vision installation is untouched, so you can test it risk free.

What’s new:

:robot: The AI assistant that writes and fixes scripts. Describe what you want, or hit Fix with AI on a failing macro — it inspects the page, writes the script, runs it, reads the log and iterates. JS is what makes this work well: the AI is far better at writing a program than at filling in a command table, and when it fails, the error tells it exactly what went wrong. Soon: UI.Vision speaks MCP, so Claude Code (or any MCP client) can drive your side panel directly, with your project context behind it (details follow).

:computer_mouse: Real browser clicks from script that (unlike ) require no focus. uiv.browser.click, .move, .type — genuine mouse and keyboard input inside the browser, no XModule needed, and they hit whatever you can find: a locator, an image, OCR text, or coordinates. Because targets are now composable in JS, “click 40px right of that label” or “find inside this area only” is one expression instead of a special command variant. Chrome and Edge, also with the window in the background; Firefox runs everything else.

:memo: Macros can now be JavaScript. A new macro kind, next to the classic table: plain modern JS with a uiv.* API — uiv.click, uiv.type, uiv.findImage, uiv.ocr.findText, uiv.csv.*. Real if/for/try/catch instead of while_v2 and !statusOK juggling, real exceptions with real messages, breakpoints and step-through. Anything without a JS equivalent still runs via uiv.run('command','target','value'), so nothing is out of reach. All demo macros ship in both flavors so you can read them side by side. Scripts also run 50–300× faster in tight loops than the old interpreter.

Feedback, bug reports and opinions on are very welcome.

2 Likes
3

tried;

scenario:

1.don’t see the loop function and the “Play from here” command as previously;

2.the new ”Btype” worked, while ”Xtype” not work with ${KEY_ESC} (XModule insalled);

question:

1. where are they? missing in this version?

2. why ”Xtype” not work with ${KEY_ESC}, which is known as real user simulation?

expectation:

1.have them as before;

2.have the logic understood;

PS: other points, not so fancy currently to have personally,

thanks.

4

  1. Loop function is removed, as the script can do it easily now. In the AI chat you can just say “Make the macro loop 10 times”, and the loop will be added. Typically the AI chat will add something like for (var i = 1; i <= 10; i++) {....}

  2. Play from here is still there, in the sidebar you need to have the “Dev Mode Switch” on to see it:

Run from here
Run from here720×1450 130 KB

In IDE as always:

Play from here = Run from here
Play from here = Run from here500×669 83.3 KB

We only kept the “Run from here with vars”, as the plain Run from here with reset vars seems rarely used/needed.

5

”Xtype” not work with ${KEY_ESC},

We will test it. If it does not work, that would be a bug, and will be fixed.