With the update to its coding agent Jules, bringing it to the command line and offering an API, Google hopes to make its agents deeply integrated into workflows.

Gemini analyzes the scene and replies with a function call such as “click,” “type,” or “scroll,” which the client executes.