UI Vision use macros control some terminal base application (e.g. AS400 RPG)

Ui.Vision
1

Can you UI Vision RPA use macros control some terminal base application (e.g. IBM AS400 RPG)?

image
image.png891×540 35 KB

2

Yes you can must use desktop automation

https://ui.vision/rpa/docs/visual-ui-testing

3

To get the text from the AS400 screen use OCR screen scraping:

The commands OCRExtract and OCRExtractRelative will be useful, see OCR Screen Scraping with built-in OCR. Free RPA software for Windows, Mac and Linux.

I did a quick test with the online ocr (same OCR engine as ui.vision!), and the text font and numbers are recognized very well, as the overlay shows:

as400%20ocr
as400%20ocr1605×981 199 KB

4

Very Nice. It works basically to read data. I have further question that
can imacros can simulate input in User ID and Password? e.g. mouse click to the boxes to input user id CWXJJ and password 123456 Please see the image

image
image.png1239×543 122 KB

5

…simulate input in User ID and Password?

Yes, UIvision can do this. The challenge is to get the cursor in the right place. To do this, there are two good options:

  • Relative Clicks with XClickRelative. It allows you to tell the RPA software to click e. g. to the right of a certain word or image. In your case this means to find the image for “User ID:” and then click in the box to the right of it.

  • Use XTYPE ${KEY_TAB} or ${KEY_ENTER} to move between input boxes. If your input screen supports TAB to move between the input fields, this is allows for a very fast and reliable automation.

  • Once the focus (cursor) is in the right field, use XType to enter the text.

This is how your XClickRelative input image could look. It finds the image inside the green frame, and then clicks in the middle of the pink frame:

relative%20click
relative%20click776×126 18.5 KB

6

Thanks very much. I can use Relative Clicks and XTYPE to perform this in desktop.
My code is :
{
“Name”: “TestTerm1”,
“CreationDate”: “2020-2-26”,
“Commands”: [
{
“Command”: “XClickRelative”,
“Target”: “inputterm1_dpi_96_relative.png”,
“Value”: “”
},
{
“Command”: “XType”,
“Target”: “{KEY_P}{KEY_E}{KEY_R}{KEY_C}{KEY_Y}{KEY_F}{KEY_O}{KEY_N}{KEY_G}", "Value": "" }, { "Command": "XType", "Target": "{KEY_TAB}”,
“Value”: “”
},
{
“Command”: “XType”,
“Target”: “{KEY_P}{KEY_I}{KEY_G}{KEY_P}{KEY_I}{KEY_G}{KEY_5}", "Value": "" }, { "Command": "XType", "Target": "{KEY_ENTER}”,
“Value”: “”
}
]
}

7

But I want to do enhancement further that I copy the User ID and Password from excel and paste to the cursor place. That means don’t want to use {KEY_P}{KEY_E} … to input text. Is it possible or any suggestions?

{
  "Command": "XType",
  "Target": "${KEY_P}${KEY_E}${KEY_R}${KEY_C}${KEY_Y}${KEY_F}${KEY_O}${KEY_N}${KEY_G}",
  "Value": ""
}
8

You can simply use

{
  "Command": "XType",
  "Target": "Your text here",
  "Value": ""
}

You need the ${KEY_P} notation only if you explicitly want to simulate keyboard presses. For normal text entering XType | text is fine.

See also:

The screencast shows (among other things) the difference between sending text strings like XTYPE | abc and explicitly simulating keyboard key presses with XTYPE | ${KEY_D}${KEY_E} - both work, but only KEY_D turns the keyboard buttons green.

9

For normal text entering XType |“Your text here”
{
“Command”: “XType”,
“Target”: “PERCYFONG”,
“Value”: “”
}
No works.

image
image.png1176×566 77.9 KB

10

I tried your vbs method : keypress1.vbs. Also not work. For WScript.Shell, do we need to install something?

keypress1.vbs code :
Set WshShell = WScript.CreateObject(“WScript.Shell”)

WshShell.SendKeys “PERCYFONG”

11

This is an interesting result. I use XType for Citrix automation, and there xtype | text works. But since the VBS sendkey command fails, too, it seems the AS400 terminal needs the full keystroke “keypress” simulation.

In your case what I would do is to recreate a small script (e. g. Javascript…) that converts “hello” to ${KEY_H}${KEY_E}${KEY_L}${KEY_L}${KEY_O} and so on. Once you have automated that part once, you can use it for all your AS400 input fields.

12

Another idea: Does copy & paste work at the terminal?

If so, you can use this:

  • Store | your text here | !clipboard
  • XType | ${KEY_CTRL+KEY_V}

=> Just paste the text into the field(s)

13

Try to move in field with xtype and ${KEY_TAB} is usually detect filed with KEY_TAB after use Xtype and your TEXT

14

I am not sure that Citrix automation is running your desktop application （ie terminal based as400）in citrix client and then you can use desktop automation to control the input like using xtypel”Percyfong” instead of using fullkeystroke. Please advise.