Batch processing, postgresql, and linux (or windows)

Ui.Vision General Discussion
1

Hi folks. I’ve been looking at implementing a solution with UI Vision for quite a while and I’d like to see if the community thinks this is feasible/how feasible it is.

UI Vision running on Linux (with Chrome or Firefox), or Windows (with Chrome or Firefox)
Preference is Linux…but on Fedora I can’t seem to get Xmodules installed/working properly.

  1. Customers request a custom quote from our company
  2. The quote goes into the database as a unique entry ‘submitted’, fires off a trigger to create a CSV
  3. The CSV is generated on the server, it has a unique ID and a ‘submitted’ status as a field in the CSV.
  4. A script on the Linux or Windows machine is polling the server, looking for CSVs. When there’s one there, it pulled it over to do work.
  5. The script copies the CSV into the datasources directory and launches UI Vision against the CSV.
  6. UI Vision does a csvread and fills a form on a website based on the values in the CSV.
  7. UI vision submits the form, receives a response, and records the quote values returned from the webpage, and writes those into a CSV to be sent back to the db for insertion or local script does the psql insert into the database.
  8. The work is complete, the values inserted, a trigger changes the flag from submitted to “processed” meaning that CSV generation in step 2 will never be processed again.

I’ve created something similar in the browser/local storage version of UI vision, but is this too complex for UI Vision outside of the local storage or is this a pretty standard usage?

Thanks,
Maine

2

I am not sure I understand the question. So you created this solution already with the free version? Nice, then it will work also with the PRO version with local storage.

3

I did something less complex (read csv, open website, login to website, UIV fills out form, submits form, scrapes response dom, creates new csv)… just seeing if anyone else has done something similar… I see 0 posts talking about postgresql, so just not sure if my linking with an external database besides csv’s will go as smoothly. I guess I’ll give it a shot.

4

You can leverage bash to execute a psql command to update back to the db via xRunAndWait.

The example python play_and_wait script with the psycopg2 library can do the same. Batch and queue processing can be managed by the pq library.

1 Like
5

Thank you @t.brown I will look into that! Sounds like exactly what I need.
Appreciated.