Run apple script to cycle through app windows

Ui.Vision HowTo
, , ,
1

I’m trying to cycle through a desktop app’s open windows to find a specific window. I’m running UI.Vision RPA in Chrome on macOS Monterey (the app is Java).

Xtype and cliclick both do not support the require key combination which is [cmd]+[keycode 50] (i.e. the ’ and ~ key, right above tab). I can use apple script in a terminal session to simulate the key combo. On first run attempt the system prompts to grant ‘Automation’ privileges to Terminal in Privacy and Security preferences. Once that’s done it works from then on. The command is:

osascript -e ‘tell application “System Events” to key code 50 using command down’

When I try to run the same command using XRunAndWait, it fails with this error:

“[error] Line 3: Running the process is failed.”

When I change to XRun, the macro completes but appears to do nothing, i.e. the system does not select the other open Chrome window as expected.

I suspect it’s a privileges issue but there is no option to manually add an app in the Automation privileges pane and the system did not prompt as it did when the command was run in Terminal.

Any suggestions re how to get this working, or another way to cycle through an App’s windows on macOS using automation, would be greatly appreciated.

2

I agree. Whatever the RPA software starts with XRUN runs with the limited privileges of the web browser.

(1) One solution is to run the browser itself has admin. - does this solve the issue?

(2) Another option would be to start the apple script independently and make it wait for a file to appear. And XRUN does nothing else than triggering a 2nd (tiny) script that writes this file. Once the file appears, the admin script does its work (in your case this would be sending the keystrokes).

Here is the same approach shown on Windows: XRUN calls a VBS file that does nothing else than writing a “abc.txt” file. The Powershell script runs as admin and waits for this file to start its admin level work.