Converting from IMacros to UIVision

databasepro · January 26, 2024, 7:55pm

I am a long time user of IMacros. I need to convert one macro to UI Vision.

Here is what the macro does:

starting at this webpage: Bill Search - Louisiana State Legislature select “search by instrument range” and then “search”… (I do not think there is a way to get the macro to open this page. But maybe that can be worked out later)
these are legislative instruments. I need UI Vision to go bill by bill saving the things displayed on the screen and the hyperlink which is marked “more” on the screen.
the final output should be a csv file with one line per bill

My old IMacros code is somewhat inefficient but worked. Here is the IMacros code that does this for one bill assuming that the above referenced page is open in the browser. However, in the actual script, I would need a loop to go bill by bill until it reaches 100 (which I have set the website to display each time). And then if there are additional pages, to go to the next page until all pages are complete.

Thanks in advance for any help anyone can provide.

TAB T=1
TAB CLOSEALLOTHERS
SET !EXTRACT_TEST_POPUP NO
SET !REPLAYSPEED FAST
SET !PLAYBACKDELAY 0

TAG POS=1 TYPE=A ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_HyperLink1 EXTRACT=HREF
TAG POS=1 TYPE=A ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_HyperLink1 EXTRACT=TXT
TAG POS=1 TYPE=A ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LinkAuthor EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelKWordAndSTitle EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelStatus EXTRACT=TXT
TAG POS=1 TYPE=SPAN ATTR=ID:ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelConsidered EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=mytable.csv

Plankton · January 28, 2024, 7:31pm

Converting this iMacro macro to ui vision should be straightforward. Some starting points:

To extract the URL, use storeattribute

To extract text, use storeText

Ui vision has many web scraping commands.

To save the extracted data to a csv file use

storeText | xpath=YourXpathHere | !csvline
storeText | xpath=AnotherTextXpath | !csvline
csvSave | mytable.csv

csvSave saves the values of the !csvline internal variable to a csv file. It creates one line per csvSave call. So it does the same as the SAVEAS TYPE=EXTRACT thing.

For the loop, have a look at the TIMES command.

Selectors:

A tricky change is that you need to replace the iMacros ATTR=ID:ctl00_ctl00_Page..._ctrl0_HyperLink1 thing with a regular Xpath as used by Selenium IDE, Ui.Vision and most other tools.

Method 1: Ui.Vision uses XPaths for element selection, so you’ll need to adjust the XPaths. In this case you can try to simply use (//*[@id='...'] ).

So for example it would be storeText | xpath=//*@id=ctl00_ctl00_Page..._HyperLink1] | !csvline to match the specific element in your web page.

Method 2: Alternatively, you can simply re-record the sequence with Ui.Vision. The app will then create CLICK commands with the Xpath selectors for you Then, replace the CLICK command with the storeText command (= both use the same selector).

If you get stuck at some point during the imacros to ui.vision conversion, just ask again

databasepro · January 28, 2024, 8:04pm

Thank you so much, Plankton. I greatly appreciate your response.

Since I posted this, I have gotten quite far in getting it to save one or two entries from the website to a csv file. But I definitely need some refinements.

To start with, how do you get it to dump the csvsave results in the browser? The saved csv file has the results of each time I have run the macro. I need to dump the cache before starting to run the macro again.

Thank you.

Plankton · January 29, 2024, 8:16am

Just to clarify, you need something like the iMacros CLEAR command?

(imacros) CLEAR:

Clears the browsers cache and all cookies. Can be useful, for example, to delete Web site cookies so every macro run starts at the same point. It is also useful to use this command before doing website response measurements.

The Ui.Vision equivalent would be the deleteAllCookies command.

For something more powerful, see How to programmatically clean cookies and the browser cache and autocomplete and the browsing history.

Yet another cache clearing option is to tell Chrome to clear everything on exit.

Plankton · January 29, 2024, 8:23am

Oh, upon re-reading your question: Is your question how to remove the the CSV file and start a fresh one? That is answered in the linked forum post

databasepro · January 29, 2024, 1:07pm

Thank you, Plankton.

My goal is to start a fresh csvsave so that data I collected on previous runs does not appear in the new run. I tried the delete All Cookies and then had it save the data. This didn’t work for me. perhaps I am doing something wrong. I put this command as the first one in my script.

Removing the previous csvfile from my files is not a problem.

Plankton · January 29, 2024, 9:08pm

Usually once you use csvSave, the content of the !csvline variable is reset/cleared.

In other words, you need to use csvSave once for each row in the CSV file.

databasepro · January 30, 2024, 1:12am

Thank you, Plankton.
So I don’t think I am explaining the problem adequately.
Let me try again:
for each line in the csv, I have a csvsave command
once I have gathered data for each of the csv records I issue the localstorageexport command which allows me to download the file

The problem is that the csv which is downloaded has every entry from every try when I have run this macro. It even has entries from when I ran the DemoExtract macro! I tried using the delete all cookies command at the beginning of the macro and that doesn’t work.

Posting what I have so far so you can see. Hopefully I am doing a better job of explaining the problem. I have a lot more work to do to get this running as my old IMacros script and I am just learning a little about UI.Vision. I know that the below code is inefficient but just trying to get something to work for now.
Thank you again for your help.

{
“Name”: “Test2Good Working Copy”,
“CreationDate”: “2024-1-29”,
“Commands”: [
{
“Command”: “open”,
“Target”: “Home - Louisiana State Legislature”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “click”,
“Target”: “linkText=2024 Regular Session”,
“Value”: “”,
“Targets”: [
“linkText=2024 Regular Session”,
“xpath=//[@id="ctl00_ctl00_MainMenun9"]/td/table/tbody/tr/td/a",
“xpath=//div[2]/table/tbody/tr/td/table/tbody/tr/td/a”,
“css=#ctl00_ctl00_MainMenun9 > td > table > tbody > tr > td > a”
],
“Description”: “”
},
{
“Command”: “selectWindow”,
“Target”: “tab=1”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “click”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ddlInstTypes2”,
“Value”: “”,
“Targets”: [
“id=ctl00_ctl00_PageBody_PageContent_ddlInstTypes2”,
“name=ctl00$ctl00$PageBody$PageContent$ddlInstTypes2”,
"xpath=//[@id="ctl00_ctl00_PageBody_PageContent_ddlInstTypes2"]”,
“xpath=//select[@id=‘ctl00_ctl00_PageBody_PageContent_ddlInstTypes2’]”,
“xpath=//select”,
“css=#ctl00_ctl00_PageBody_PageContent_ddlInstTypes2”
],
“Description”: “”
},
{
“Command”: “clickAndWait”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_btnSearchByInstRange”,
“Value”: “”,
“Targets”: [
“id=ctl00_ctl00_PageBody_PageContent_btnSearchByInstRange”,
“name=ctl00$ctl00$PageBody$PageContent$btnSearchByInstRange”,
“xpath=//*[@id="ctl00_ctl00_PageBody_PageContent_btnSearchByInstRange"]”,
“xpath=//input[@id=‘ctl00_ctl00_PageBody_PageContent_btnSearchByInstRange’]”,
“xpath=//td/div/input[3]”,
“css=#ctl00_ctl00_PageBody_PageContent_btnSearchByInstRange”
],
“Description”: “”
},
{
“Command”: “deleteAllCookies”,
“Target”: “”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_HyperLink1”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeAttribute”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_HyperLink1@href”,
“Value”: “b”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “Home - Louisiana State Legislature”,
“Value”: “a”,
“Description”: “”
},
{
“Command”: “executeScript_Sandbox”,
“Target”: “return ${a} + ${b}”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelStatus”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelConsidered”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LabelKWordAndSTitle”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl0_LinkAuthor”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “echo”,
“Target”: “${!csvLine}”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “comment”,
“Target”: “Append content of !csvLine to CSV file (or create file if none exists)”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “csvSave”,
“Target”: “CurrencyConverterData”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_HyperLink1”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeAttribute”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_HyperLink1@href”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_LabelStatus”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_LabelConsidered”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_LabelKWordAndSTitle”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “storeText”,
“Target”: “id=ctl00_ctl00_PageBody_PageContent_ListViewSearchResults_ctrl1_LinkAuthor”,
“Value”: “c2”,
“Description”: “”
},
{
“Command”: “store”,
“Target”: “${c2}”,
“Value”: “!csvLine”,
“Description”: “”
},
{
“Command”: “echo”,
“Target”: “${!csvLine}”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “comment”,
“Target”: “Append content of !csvLine to CSV file (or create file if none exists)”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “csvSave”,
“Target”: “CurrencyConverterData”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “comment”,
“Target”: “If needed, you can download (save) the CSV data from the CSV tab to the the download folder”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “localStorageExport”,
“Target”: “currencyconverterdata.csv”,
“Value”: “”,
“Description”: “”
}
]
}

Plankton · January 30, 2024, 12:27pm

If you add #DeleteAfterExport in the 3rd column, then UI.Vision RPA deletes the file after the export. So this should work:

localStorageExport | currencyconverterdata.csv | #DeleteAfterExport

databasepro · January 30, 2024, 1:22pm

Thank you, that worked perfectly!

I have more questions, should I continue on this thread or start a new one?

Plankton · January 30, 2024, 4:40pm

Great!

Best to start a new thread for each question.