Ui.Vision Desktop Automation is the new generation of the XModules. Now it is one app instead of four, faster start, DPI-aware image search, multi-monitor support and better in-app OCR.
You only need this if your macros use desktop-scope features (uiv.desktop.*,scope: “desktop”, XClick / XType / XRun and hard-drive file storage). A browser extension cannot do these things on its own; this helper does them for it. Pure in-browser macros need nothing installed.
The new module works with both, new JS macros and classic table macros. The only feature not in the new XModule is the green/pink image patterns, but JS has very good replacements for this.
Multi-monitor support: Ui.Vision looks always at the screen where its browser is. It can the main, 2nd, 3rd etc monitor . If you are unsure what your Ui.Vision sees, go to the Desktop Automation settings tab and use the “Test Desktop Screenshot” button near the bottom.
Beta testing: There is a switch in the extension, so you can switch between the current Version 1 and the new Version 2. If you only have the new Version 2 installed, this switch does not show.
How to install Ui.Vision Desktop Automation for Linux
Requirements
64-bit Linux, glibc 2.35 or newer (Ubuntu 22.04+, Debian 12+, Fedora 36+,
and anything newer). Check with: ldd --version
Wayland or X11 both work! On Wayland the system shows a one-time
“Share your screen” prompt the first time a macro captures the screen or
moves the mouse (see “First run” below) — this is the OS asking your
permission, and it is remembered afterwards.
Install
Extract this tarball anywhere:
tar -xzf uivision-desktop-automation-linux-x64.tar.gz
cd uivision-desktop-automation
Run the installer (NO root / sudo needed — it installs into your home
folder only):
./install.sh
It copies the helper to ~/.local/share/uivision/ and registers it with
Chrome, Edge, Chromium and Firefox.
Fully quit and reopen your browser, so it picks up the new helper.
First run — grant permission (Wayland only)
The first time a macro does anything on the desktop (a screenshot, a click, a
keypress), your desktop shows a “Share your screen” / “Remote desktop” dialog.
Choose your monitor and click Share/Allow ONCE. The permission is remembered,
so later runs do not ask again. On X11 there is no such prompt.
Verify it works
In the browser: open the UI.Vision side panel, go to
Settings > Desktop Automation
and click:
“Test native host” → should report the app version (green).
“Test desktop screenshot” → should show a screenshot of your screen.
“Test read/write” → confirms the home directory is writable.
If all three are green, you are ready. Try the demo macros in
“Demo and QA Test Scripts > Desktop Automation”.
Uninstall
./install.sh --uninstall
Removes the helper and all the browser registrations.
Troubleshooting
“Not installed” after install.sh: you did not fully restart the browser.
Quit it completely (all windows) and reopen; a new tab is not enough.
Snap or Flatpak browser (the default Firefox on Ubuntu is a Snap): sandboxed
browsers reach native helpers only through a portal, and per-user native
messaging manifests may not be visible to them. Use a non-Snap browser
(Chrome/Edge .deb, or Firefox from mozilla.org) for desktop automation, or
see the online docs for the portal workaround.
The screen-share prompt never appears / capture fails on Wayland: make sure
xdg-desktop-portal and the backend for your desktop are installed
(xdg-desktop-portal-gnome on GNOME, xdg-desktop-portal-kde on KDE, or
xdg-desktop-portal-wlr on wlroots compositors).
. If you are unsure what your Ui.Vision sees, go to the Desktop Automation settings tab and use the “Test Desktop Screenshot” button near the bottom.
To get started, you need this Ui.Vision BETA version for the extension (the regular store version does not support the new XModules yet)
