Gemini: Type a text store in a box

UI.Vision RPA HowTo
1

Hi, i finally know how to storage a text but… Now don`t know how to paste in a browser text box.

this is the error:

  • Executing: | type | ${OA} | |

  • [error]

Line 11: timeout reached when looking for element ‘Escuchar cualidades del sonido (altura, timbre, intensidad, duración) y elementos del lenguaje musical (pulsos, acentos, patrones, secciones) y representarlos de distintas formas.’

Thanks.

PD: The text in Italic = OA

2

Hi, do you have a link to the website?

3

Hi :wave: https://gemini.google.com/app

4

Indeed, the regular TYPE command does not work for Gemini automation/chatbox.

Solution:

  • Replace CLICK with XCLICK

  • Replace TYPE with XTYPE

Then it works:

5

Hi, tried two different ways, it doesn`t work :frowning:

{
“Name”: “gemini”,
“CreationDate”: “2024-2-16”,
“Commands”: [
{
“Command”: “open”,
“Target”: “https://gemini.google.com/app”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “XClick”,
“Target”: “1rnwb3_dpi_159.png”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “XType”,
“Target”: “hola”,
“Value”: “”,
“Description”: “”
}
]
}

{
“Name”: “gemini”,
“CreationDate”: “2024-2-16”,
“Commands”: [
{
“Command”: “open”,
“Target”: “https://gemini.google.com/app”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “XClick”,
“Target”: “1rnwb3_dpi_159.png”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “XType”,
“Target”: “hola”,
“Value”: “”,
“Description”: “”
}
]
}

Video link: Gemini

6

sorry this is the second

{
“Name”: “gemini”,
“CreationDate”: “2024-2-16”,
“Commands”: [
{
“Command”: “open”,
“Target”: “https://gemini.google.com/app”,
“Value”: “”,
“Description”: “”
},
{
“Command”: “XClick”,
“Target”: “xpath=//[@id="app-root"]/side-navigation-v2/bard-sidenav-container/bard-sidenav-content/div/main/div[2]/chat-window/div/div[2]/div/input-area-v2/div/div/div/div/div/rich-textarea/div/p",
“Value”: “”,
“Targets”: [
"xpath=//[@id="app-root"]/side-navigation-v2/bard-sidenav-container/bard-sidenav-content/div/main/div[2]/chat-window/div/div[2]/div/input-area-v2/div/div/div/div/div/rich-textarea/div/p”,
“xpath=//rich-textarea/div/p”,
“css=#app-root > side-navigation-v2 > bard-sidenav-container > bard-sidenav-content > div > main > div.content-container.ng-tns-c3840756465-0 > chat-window > div.chat-container.ng-tns-c3171017574-1.at-least-desktop-small.ui-v2-enabled.ng-star-inserted > div.bottom-container.ng-tns-c3171017574-1.ui-v2-enabled.ng-star-inserted > div.input-area-container.ng-tns-c3171017574-1.ng-star-inserted > input-area-v2 > div > div > div.text-input-field_textarea-wrapper.ng-tns-c2876043874-3 > div > div > rich-textarea > div.ql-editor.ql-blank.ui-v2-enabled.textarea > p”
],
“Description”: “”
},
{
“Command”: “XType”,
“Target”: “hola”,
“Value”: “”,
“Description”: “”
}
]
}

7

What error message do you get? Can you post a video of the macro running?

9

no error…

10

Hola! My guess is that the XClick Xpath is wrong. It is usually much better to use XClick + IMAGE or XClickText + Text.

(1) So one option is to solve this by using an image instead:

XClick | image of gemini prompt

(2) Here is a solution with text and local OCR (works for me!):

{
  "Name": "Gemini RPA",
  "CreationDate": "2024-2-22",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://gemini.google.com/app",
      "Value": "",
      "Description": ""
    },
    {
      "Command": "XClickText",
      "Target": "prompt",
      "Value": "",
      "Description": "Find the input field with OCR. Of course, for Spanish, use a Spanish word, just as is written inside the Gemini box. Use ONE word only."
    },
    {
      "Command": "XType",
      "Target": "works",
      "Value": "",
      "Description": "send text"
    }
  ]
}
11

Gracias! It works with XClick + IMAGE