[ui.vision support Q0002] How to type a command inside iterm2 and detect its output?

OS: macOS catalina 10.15.x
XmodulePRO: yes

The steps I want to perform are:

  1. open iTerm2
  2. run the command node -v
  3. retrieve the output regardless what it says

I can now do step 1.

How to do step 2 and 3?

What it looks like when I do this manually

image

Step 2: Send XType | node-v ${KEY_ENTER}

Step 3: Is the text always just one line?

I am unsure.

Is there a way to write it so that I don’t have to care either way?

image

iTerm2 opens but nothing gets typed out

I have already given kantu-xy-host accessibility permissions

Here is a screencast. I used OCR for screen scraping:

ok

Let’s say we ignore step 3 for now.

Let’s focus on step 2. Because I cannot even get that to work.

You started with XClick. I use XRun for step 1.

Does that matter? Because for me, my step 2 XType cannot work at all.

SEe my gif

It should not matter.

Question: Does the Demo-XClick macro work? Because if not, then this is the solution:

Yes, it already was

Kinda

SEe attached

Your screenshot is missing the browser part. => Did Demo-XClick draw anything in the canvas?

The error I see in your screenshot is just a certain image not found, this can be due to different resolutions on your Mac. All demo images were created on Windows 96dpi machine.

Yes i think so.

I’m sorry. I don’t want to be rude. but my main focus is getting the iterm2 to work. I’m less interested to make Demo-XClick to work.

I think Demo-Xclick demonstrated that my laptop can work with XClick. even though there’s one error message. Is my understanding correct?

If so, I want to get back to the focus on iTerm2. What’s stopping XType from working on this?

And you did mention it should not matter whether I use XRun or XClick… so what’s the issue here?

and can you paste your XClick command target and value here? so i can experiment with using it instead of XRun. Thank you

The challenge is, in our test with the console all works well (see the screen video above). So the question is why does it not work on your machine?

Some ideas:

  • Permission issue. XClick and XType use the same module. Thus my questions about XClick. But you said, XClick in general works on your Mac?

  • Test: Does XType work with other apps, e. g. Send text to Notes?

  • Lets ignore XRun vs XClick for now and make this small test macro by manually launching iterm2:

Adding this between the XRun and the XTYpe works.

Curious… but … Why?

Ah, this is a good find. It explains the issue:

  • XRun starts the app, and then, before the console is completely loaded and ready to receive keystrokes, XType sends the keystrokes => they go nowhere (probably to the desktop).

  • The solution is to use either PAUSE | 3000 (as you did)
    or add XClick | Image of console titlebar (or similar) before using XType.

1 Like