Thoughts of LLM and Ui.Vision Integration Test

A user recently asked us an interesting question: Can I integrate OpenAI’s operator with your system? My automation system is meant to integrate the operator with Instagram to receive leads interested in scheduling appointments at my clinic. I need an RPA (Robotic Process Automation) that interacts with the operator, connects to my clinic’s proprietary system (which doesn’t have an API), checks available time slots in the calendar, offers them to the lead via the operator, and then, once the lead confirms a time, automatically books it and marks that slot as unavailable in my cloud-based proprietary calendar system. Can your system achieve this to make the scheduling process 100% automated?

Our suggestion is to first try to achieve the desired result using Anthropic Computer Use that is currently available inside Ui.Vision.

Once that works, if needed: The second step would be to optimize the solution for cost (e. g. switch to OpenAI’s operator + Ui.Vision? We plan test that soon.) Or optimize for privacy and use local LLMs.