QUERY: Can a RPA macro capture website info (table scraping)?

UI.Vision RPA HowTo
, ,
2

Short answer: Yes, Kantu can do that. For web scraping the storeText, storeValue and sourceExtract commands are very useful.

How to do it:

  1. Record a macro, and click on the data that you want to save. This creates the right locator to find the data. Then change click to storeText.

extract1
extract11099×422 61.1 KB

  1. If you want to save to a CSV file, use the special internal variable !csvLine with storeText. Everything that gets assigned to !csvLine will get saved as new column to a CSV file, once csvSave is called.

extract%20with%20storetext
extract%20with%20storetext1224×836 83.9 KB

  1. That is all! Now, if you want to save the csv file to disk, too, then call localStorageExport ibandata.csv

What is missing in this flow is the logic that navigates to each iban data page. To create that part of the macro you can click on few of the country flags and will see that their locator is as follows:

//*[@id="country-list"]/li[1]/a/img (first flag)
//*[@id="country-list"]/li[2]/a/img (2nd flag)
//*[@id="country-list"]/li[3]/a/img (3rd flag)

=> so we replace the 1,2,3… with the !loop variable:

//*[@id=“country-list”]/li[${!loop}]/a/img

Done :slight_smile: . Here is a screencast of the macro running. Note that is is started via the Loop button:

Web scraping with storeText

And here is the macro:

{
  "CreationDate": "2018-7-4",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://www.xe.com/ibancalculator/countrylist/",
      "Value": ""
    },
    {
      "Command": "clickAndWait",
      "Target": "//*[@id=\"country-list\"]/li[${!loop}]/a/img",
      "Value": ""
    },
    {
      "Command": "storeText",
      "Target": "//*[@id=\"contentL\"]/div[1]/table/tbody/tr[1]/th[2]",
      "Value": "!csvLine"
    },
    {
      "Command": "storeText",
      "Target": "//*[@id=\"contentL\"]/div[1]/table/tbody/tr[2]/td[2]",
      "Value": "!csvLine"
    },
    {
      "Command": "storeText",
      "Target": "//*[@id=\"contentL\"]/div[1]/table/tbody/tr[3]/td[2]",
      "Value": "!csvLine"
    },
    {
      "Command": "csvSave",
      "Target": "ibandata",
      "Value": ""
    }
  ]
}
3 Likes