Xtype sending keys to itself

UI.Vision RPA Bug Reports
1

I’m working on Ubuntu and Firefox. All is working but Xtype send keys to its window. I need to copy the browser page and paste it in Word, but when I do:
Xtype | ${KEY_CTRL+KEY_A}
the UI Vision page itself is selected and not the browser one
I even tried with KEY_CTRL+KEY_S, but the same happens
What am I doing wrong?

Schermata da 2021-03-01 00-17-46
Schermata da 2021-03-01 00-17-461920×1080 355 KB

2

You need to bring focus by using bringbrowsertoforeground command

3

It seems it doesn’t work for me. Adding bringbrowsertoforeground command only changes in that the keystroke is not sent to the UI windows itself but is lost at all (i.e. it doesn’t do anything)

4

Admin Edit 2021/10/18: We found an easier workaround to make bringbrowsertoforeground work on Linux+Firefox, see below

This is Linux and Firefox, or? bringbrowsertoforeground does not work there.

Solution:

  1. Use Chrome, bringbrowsertoforeground works fine in Chrome on Linux

  2. Or use xdesktopautomation | true + one XClick on e. g. the title bar of the browser, that brings Firefox to the foreground, too.

5

Sorry if I wasn’t clear enough… The problem is about Xtype not sending keystrokes to the browser window.

Using the “bringbrowsertoforeground” command didn’t help, but I didn’t mean the bringbrowsertoforeground command doesn’t work, I think it does

Running a very simple macro:
pause | 5000 | |
open | https://google.com | |
XType | ${KEY_CTRL+KEY_A} | |
XType | ${KEY_CTRL+KEY_C} | |

that should copy the “google” main page into the clipboard,

  • if the focus is on the UI window, the keystorkes are sent to itself (see picture) and I find it in the clipboard
  • if the focus is on the browser window (manually clicking on it while the pause command is running or by the bringbrowsertoforeground command, it’s the same) simply nothing happens, the keystrokes are lost somewere and the clipboard is empty

I think it’s a bug, or simply it can’t run on this linux/browser. Hope this helps to understand…
Ubuntu is 20.04.2, Firefox 86.0

As far as my work is concerned, I’m finding other solutions not involving Xtype command :), but always using UI.vision, it’s great!

Best regards
Gianni

6

Thanks for the feedback. Great to hear that you found another solution!

But just for the record, we tested this further. So the issue is that in Ubuntu even a manual click inside the Firefox viewport does not bring the browser to the foreground. In Chrome it works as expected. So this is a Firefox internal bug. But copy and paste still works fine. So the browser does get the keyboard focus, even so it does not come to the front:

{
  "Name": "copy and paste1",
  "CreationDate": "2021-3-2",
  "Commands": [
    {
      "Command": "store",
      "Target": "this should not be there after copy and paste",
      "Value": "!clipboard"
    },
    {
      "Command": "open",
      "Target": "https://www.google.com/",
      "Value": ""
    },
    {
      "Command": "XClick",
      "Target": "google_dpi_96.png",
      "Value": ""
    },
    {
      "Command": "XType",
      "Target": "${KEY_CTRL+KEY_A}",
      "Value": ""
    },
    {
      "Command": "XType",
      "Target": "${KEY_CTRL+KEY_C}",
      "Value": ""
    },
    {
      "Command": "echo",
      "Target": "Clipboard=${!clipboard}",
      "Value": "green"
    }
  ]
}
7

Here is a very easy workaround for this issue:

  • First do a BringIDEandBrowserToBackground (minimizes IDE and browser)

  • and then do bringbrowsertoforeground (open browser again)

=> Now Firefox is in front, and the UI.Vision IDE no longer blocks XClick/XMove commands.

https://www.screencast.com/t/dpfIUrVlY0x

8

This solution doesn’t work for me on Ubuntu running firefox. Bringbrowsertoforeground brings both windows back up with IDE on top.