Interesting question from an enterprise license user that we want to share:
Good day! I’m writing to ask if you’ve heard of anyone who’s been able to get UIVision RPA to work with Salesforce Marketing Cloud, which I’ve seen described as being actively hostile to macros.
I have an example screen below. The red boxes are around all of the information I want Ui.Vision to test/report on, but it’s been a real struggle to get anything going.
There are multiple frames, none of this presents in the source, and none of the data is in elements with actual, you know, ID’s. Plus everything can be slightly off depending on several factors. Worse, navigating to here is difficult since the recorder doesn’t pick up any valid, reusable targets for my clicks.
Would AI help, do you think, picking up all this? Would it be able to understand/OCR the data I’m looking for. I know I could probably define explicit locations for certain things, but, then… Like “is contact ID the primary key?” would be a tough one. (ocr a checkbox?). Any suggestions?
So if sending a screenshot to a reliable third party cloud service like Anthropic is acceptable, the aiPrompt command makes the automation a breeze:
I blurred most data and then tested your screenshot with the aiPrompt command:
aiPrompt | sfmc.png#Attached a UI. Return the name of the primary ID, see the table on the right. | s
Answer: Based on the table shown in the right panel, the primary ID field is ContactId.
aiPrompt | sfmc.png#Attached a screenshot. Return the text in the "Used for sending" box.| s
Answer: Yes
aiPrompt | sfmc.png#Attached a screenshot. Return the text in the "Used for testing" box.| s
Answer: No
Note1:sfmc.png# is the screenshot from the forum post above. I imported it into the Visual tab and then ran the commands on it. During an automation you would most likely take the screenshot on the fly with capturescreenshot | sfmc.png or (re-use) the __last_screenshot.png image. The file __last_screenshot.png is created by all visual commands e. g. XClick, XClickText, visualAssert… . Therefore if you run a visual command before doing the aiPrompt, there is no need to retake the screenshot if the screen did not change.
Note2: You can use the regular Claude.ai chat (Claude 3.5 Sonnet engine) to test your prompts for use with the aiPrompt command:
