Download the Desktop Automation Module (XModules Version 2)

Announcements
1

Ui.Vision Desktop Automation (Beta) is the new generation of the XModules. Now it is one app instead of four, faster start, DPI-aware image search, multi-monitor support and improved in-app OCR.

You only need this if your macros use desktop-scope features (uiv.desktop.*,scope: "desktop", XClick / XType / XRun and hard-drive file storage). The browser extension itself cannot do these things on its own and needs this helper app.

Multi-monitor support: Ui.Vision looks always at the screen where its browser is. It can the main, 2nd, 3rd etc monitor :desktop_computer:. If you are unsure what your Ui.Vision sees, go to the Desktop Automation settings tab and use the “Test Desktop Screenshot” button near the bottom.

:backhand_index_pointing_right: To get started, you need this Ui.Vision BETA version for the extension (the regular store version does not support the new XModules yet)

Then, get the new XModules for Win, Mac or Linux:

The installers might have sightly different version numbers due to our internal built-process. Feature-wise, Ui.Vision Desktop Automation for Windows, Mac and Linux is identical.

Upgrading from Version 1? Here is some info:

XModules Version 1 is do not get overwritten! Both can be installed at the same time. There is a switch in the extension, so you can switch between the current Version 1 and the new Version 2. If you only have the new Version 2 installed, this switch does not show.

The new module works with both, new JS macros and classic table macros. The only feature not in the new XModule is the green/pink image patterns, but JS has very good replacements for this.

2

How to install Ui.Vision Desktop Automation for Linux :penguin:

Requirements

  • 64-bit Linux, glibc 2.35 or newer (Ubuntu 22.04+, Debian 12+, Fedora 36+,
    and anything newer). Check with: ldd --version
  • Wayland or X11 both work! On Wayland the system shows a one-time
    “Share your screen” prompt the first time a macro captures the screen or
    moves the mouse (see “First run” below) — this is the OS asking your
    permission, and it is remembered afterwards.

Install

  1. Extract this tarball anywhere:

    tar -xzf uivision-desktop-automation-linux-x64.tar.gz
cd uivision-desktop-automation

  2. Run the installer (NO root / sudo needed — it installs into your home
    folder only):

    ./install.sh

    It copies the helper to ~/.local/share/uivision/ and registers it with
    Chrome, Edge, Chromium and Firefox.

  3. Fully quit and reopen your browser, so it picks up the new helper.

First run — grant permission (Wayland only)

The first time a macro does anything on the desktop (a screenshot, a click, a
keypress), your desktop shows a “Share your screen” / “Remote desktop” dialog.
Choose your monitor and click Share/Allow ONCE. The permission is remembered,
so later runs do not ask again. On X11 there is no such prompt.

Verify it works

In the browser: open the UI.Vision side panel, go to
Settings > Desktop Automation
and click:

  • “Test native host” → should report the app version (green).
  • “Test desktop screenshot” → should show a screenshot of your screen.
  • “Test read/write” → confirms the home directory is writable.

If all three are green, you are ready. Try the demo macros in
“Demo and QA Test Scripts > Desktop Automation”.

Uninstall

   ./install.sh --uninstall

Removes the helper and all the browser registrations.

Troubleshooting

  • “Not installed” after install.sh: you did not fully restart the browser.
    Quit it completely (all windows) and reopen; a new tab is not enough.

  • Snap or Flatpak browser (the default Firefox on Ubuntu is a Snap): sandboxed
    browsers reach native helpers only through a portal, and per-user native
    messaging manifests may not be visible to them. Use a non-Snap browser
    (Chrome/Edge .deb, or Firefox from mozilla.org) for desktop automation, or
    see the online docs for the portal workaround.

  • The screen-share prompt never appears / capture fails on Wayland: make sure
    xdg-desktop-portal and the backend for your desktop are installed
    (xdg-desktop-portal-gnome on GNOME, xdg-desktop-portal-kde on KDE, or
    xdg-desktop-portal-wlr on wlroots compositors).

  • Still stuck: => https://forum.ui.vision - right here :slight_smile: