Usability Issues - UI.Vision RPA (5.5.7) vs. "vanilla" Selenium-IDE (v3.17.0)

Ui.Vision Feature Suggestions
1

Using Chrome …

I’m coming from using Selenium-IDE (v3.17.0) as a reference. The enhanced features in UI.Vision RPA are what has me jumping ship. Unfortunately, I found Selenium-IDE to provide a more “stable” environment, as an IDE, for building the scripts. Here are some observations which have a bend of “… if RPA could be more like Selenium-IDE for these, then I think usability would be facilitated”:

  1. UI.Vision “Select” for target - to select by mouseclick on the live webpage - provides a different list for Selenium-IDe vs. UI.Vision. Selenium’s return values are “fuzzier” - allowing for some “better” choices than UI.Vision gives.

UI.Vision:

css=#root > div > main > div > div.ydf-toolbar-wrapper > div.ydf-toolbar.react-draggable.react-draggable-dragged > div.ydf-toolbar-item.ydf-users-list > span
-or-
xpath=//*[@id=“root”]/div/main/div/div[3]/div[1]/div[8]/span

Selenium-IDE:

css=.ydf-users-list > span (css:finder)
-or-
xpath=//div[@id=‘root’]/div/main/div/div[3]/div/div[8]/span (xpath:idRelative)
-or-
xpath=//div[3]/div/div[8]/span (xpath:position)
-or-
xpath=//span[contains(.,‘Users list’)] (xpath:innerText)

… and having the choices available helps to accommodate for occasions when something completely relative like the xpath:innerText would help when “Users list” can be one of many menu items - but not necessarily ever in the same position. And although I can “key these in” - having the select feature give more drop-down options speeds up the workflow and use.

  1. Speaking of select, when I use the “Select” button in UI.Vision, I have to “alt-tab” to pull up the web page that UI.Vision is associated with then start selecting. In Selenium-IDE, pressing the “select” button will automatically give focus to the web browser associated with Selenium so I can immediately start selecting an element of choice. The Selenium policy is a faster workflow and helps reduce the ambiguity of “which browser window was it, now?”

  2. Speaking of the dropdown list for “Target” - in UI.Vision, I cannot select text while the dropdown box/listing is showing. So if “Select”, for example, had two options - I can’t cut/paste (from UI.Vision) these two choices for saving them off. I have to instead first select the one I would want to copy, then select another command, then re-select the command that previously had a dropdown list. Now there’s no dropdown list in UI.Vision, and I can select/copy/edit the text. Selenium-IDE will present a dropdown list - but will also allow for selecting/copying/editing the target immediately.

Not only this, but Selenium-IDE will “remember” its dropdown list of choices for Target - while UI.Vision RPA will “forget” all the choices except for the last selected item when moving off to a different command. The Selenium policy helps when its debug time to maybe try some alternate choices if something goes wrong (like, at first, choosing an element with too specific of an absolute match - when one of the other dropdown choices could be more relative and appropriate. And having these choices still available later helps expedite the debug process).

And … also on the subject of the Target dropdown list - Selenium IDE’s dropdown list can display the full text for every item in the list. They use multi-line per item. UI.Vision doesn’t do this - so you get a truncated list if the item(s) do not fit in the space provided.

  1. Selenium-IDE allows for adding breakpoints with a single-left-click. UI.Vision requires a right-click then choose “Add breakpoint”. This is good and bad. I got accustomed to it in Selenium-IDE but it’s also been annoying to “accidentally” add breakpoints when clicking around the interface. “On the fence” if this matters or not.

  2. Selenium-IDE has “disable breakpoints” switch.

  3. Selenium-IDE deals better with stopping scripts (play and record) in order to start recording more at an arbitrary spot.
    a) Selenium-IDE has a “record from here” option. Doing so doesn’t add an “open” command. You can pick up where you left off assuming the webpage associated with Selenium-IDE is “on the same screen” you want to interact with. In other words, system under test is currently in the right state. Which is useful. UI.Vision is hit-and-miss and “record” always adds an open command which is hardly ever what I want except if the script is empty to begin with.
    b) Selenium-IDE keeps the train on the tracks better. From a “stopped recording” state (script is populated, I stopped recording, may have interacted some with the webpage) - When I turn on recording and start interacting with the webpage, Selenium-IDE will keep adding commands if I click around the webpage. UI.Vision gets lost and no click commands are recorded.
    c) Selenium-IDE will automatically give focus to the associated webpage when I start recording under Selenium. UI.Vision doesn’t do this - similar to the “Select” feature.

  4. General UI interface is a bit clunkier when selecting/editing commands. Under UI.Vision, the yellow background is showing which command is selected. If I re-select the same command (left-click on the command with the yellow background) then UI.Vision treats this as a de-select all state. This, for me, is a bit clunkier to use because when there’s a failure - the failed line will be red background. This also means currently selected line - so often I find myself left-clicking the failed line - this deselects it altogether - then I have to re-left-click this line to “get it back” to select it again. I find myself clicking around too much because of the “wonky” (IMO) way the command selection works. Selenium-IDE will not de-select a line if you left-click on a line that’s already the selected line (which, under Selenium-IDE is blue background … and I don’t care about the color differences here). But I like how if I click on the blue-background selected item - it doesn’t do anything. The past successes and failures remain red or green (foreground, not background) under Selenium and do not change if I select them or select off of them. General navigation in Selenium-IDE is just, for me, better and seems more “stable”.

Certainly there’s a longer list of differences - but these are the usability ones I’ve run in so far that make general use a bit of a stick-in-the-mud. The other features I’m jumping ship for like !statusOK, screen grabs, etc. are with the price of admission (for me) - but I think I still need to standardize on Selenium-IDE for the wider group since it’s more “point and shoot”.

Thanks for listening.

2

This is really good feedback, thanks! A more detailed response will follow once we had a change to look at the issues in more detail :slight_smile:

3
  1. UI.Vision “Select” for target - to select by mouseclick on the live webpage - provides a different list for Selenium-IDe

I agree. Expanding locator option is already on our todo list. => We will try to add it with the next update.

  1. Speaking of select, when I use the “Select” button in UI.Vision, I have to “alt-tab” to pull up the web page that UI.Vision is associated with then start selecting.

Added to todo for next update

  1. Speaking of the dropdown list for “Target” - in UI.Vision, I cannot select text while the dropdown box/listing is showing.

Hmm… this feedback I did not understand. Can you maybe add a screenshot or screencast to clarify?

4.+ 5: Breakpoints

Noted and added to todo (but probably not in the next update already)

  1. Selenium-IDE has a “record from here” option.

Added to todo for next update

6b) Selenium-IDE keeps the train on the tracks better.

This issue I could not recreate. Can you maybe add a screenshot or screencast to clarify? I tested stopping and restarting the recording in UI.Vision and it adds a new OPEN (as discussed), but otherwise it continues without issues, or?

6c) Selenium-IDE will automatically give focus to the associated webpage when I start recording under Selenium. UI.Vision doesn’t do this - similar to the “Select” feature.

Good point! Added to todo for next update

  1. If I re-select the same command (left-click on the command with the yellow background) then UI.Vision treats this as a de-select all state.

Ouch, yes… => Added to todo for next update

Thanks for listening.

This was very useful feedback! If you can provide a few more details for 3+6b, that would be helpful, thanks.

4
  1. Clarification: The “Target:” field in UI.Vision has one of two states. One state there is no dropdown list where the far-right of the text field has no control - and another state where the far-right of the text field has a control (that looks like a “V”) where pressing this control will show a drop-down list of possible targets to select. Each of these targets were detected by UI.Vision from the webpage.

The state where there is no dropdown list - the text can be freely edited. You can click+hold the mouse to highlight a portion of or the entire line - for deletion, copying, etc. You can press Ctrl+A to highlight the entire Target: field. You can press Ctrl-C to copy the highlighted “Target:” field to your clipboard. Etc.

The state where there is a dropdown list - where the far-right of the text field for “Target:” shows a “V” - you cannot click+hold to highlight the line. You can’t press Ctrl+A to highlight the line. You can’t copy the line. You cannot interact with the “Target:” line except to pick another item in the list.

In order to reproduce the finding:

a) From Demo->Core folder, select the “DemoDragDrop” script. Just picking a standard starting place without any disputed “noise”.
b) Select line 4 (dragAndDropToObject | id=one | id=bin) where target is “id=one”. Since “Select” was not used to populate “id=one” in the target field - the stock load of this macro does not show a dropdown box. The “Target:” field can be freely edited. This is good.
c) Press the “select” button, then alt-tab over to the “Frame Test Page” in the browser, then click on anything - I clicked on the yellow of Frame 1. Now the “Target:” box is no longer freely editable. It’s a fixed list. List item #1 is “xpath=/html/body” and list item #2 is “css=body”. Neither one of them can be copied or edited. The dropdown list for “Target” in UI.Vision works differently than Selenium-IDE. Selenium-IDE will show a pulldown list - but the text box remains as a field that can be edited or highlighted and copied/pasted etc.

6b) I’ll have to spend more time characterizing this for a useful response. Perhaps “record from here” would help. As well as other interface issues listed elsewhere which may have masqueraded as issues I fit under 6b’s bucket. For now - I’m using Selenium-IDE as my “daily driver” and may return with something more specific for 6b) later when I continue the work requiring UI.Vision (screenshots, etc).

Thanks for the quick response and work in reproducing these.

5

Thanks again for the details. I understood the drop down issue now :slight_smile:

If you have other/more issues/suggestions in the future please feel free to update this thread anytime.

7

Meanwhile all of the suggestion have been added. So I mark this post as resolved.

1 Like