Execute actions like mouse movements, clicks, keyboard input, and screenshots in the Bytebot desktop environment.
Request The type of computer action to perform. Must be one of: move_mouse, trace_mouse,
click_mouse, press_mouse, drag_mouse, scroll, type_keys, press_keys,
type_text, wait, screenshot, cursor_position.
Mouse Actions
The target coordinates to move to. Show coordinates properties
X coordinate (horizontal position)
Y coordinate (vertical position)
Example Request { "action" : "move_mouse" , "coordinates" : { "x" : 100 , "y" : 200 } }
Array of coordinate objects for the mouse path. Keys to hold while moving the mouse along the path.
Example Request { "action" : "trace_mouse" , "path" : [ { "x" : 100 , "y" : 100 }, { "x" : 150 , "y" : 150 }, { "x" : 200 , "y" : 200 } ], "holdKeys" : [ "shift" ] }
The coordinates to click (uses current cursor position if omitted). Show coordinates properties
X coordinate (horizontal position)
Y coordinate (vertical position)
Mouse button to click. Must be one of: left, right, middle.
Number of clicks to perform.
Keys to hold while clicking (e.g., [‘ctrl’, ‘shift’]) Example Request { "action" : "click_mouse" , "coordinates" : { "x" : 150 , "y" : 250 }, "button" : "left" , "clickCount" : 2 }
The coordinates to press/release (uses current cursor position if omitted). Show coordinates properties
X coordinate (horizontal position)
Y coordinate (vertical position)
Mouse button to press/release. Must be one of: left, right, middle.
Whether to press or release the button. Must be one of: up, down.
Example Request { "action" : "press_mouse" , "coordinates" : { "x" : 150 , "y" : 250 }, "button" : "left" , "press" : "down" }
Array of coordinate objects for the drag path. Mouse button to use for dragging. Must be one of: left, right, middle.
Keys to hold while dragging.
Example Request { "action" : "drag_mouse" , "path" : [ { "x" : 100 , "y" : 100 }, { "x" : 200 , "y" : 200 } ], "button" : "left" }
Keyboard Actions
Array of keys to type in sequence. Delay between key presses in milliseconds.
Example Request { "action" : "type_keys" , "keys" : [ "a" , "b" , "c" , "enter" ], "delay" : 50 }
Array of keys to press or release. Whether to press or release the keys. Must be one of: up, down.
Example Request { "action" : "press_keys" , "keys" : [ "ctrl" , "shift" , "esc" ], "press" : "down" }
Delay between characters in milliseconds.
Example Request { "action" : "type_text" , "text" : "Hello, Bytebot!" , "delay" : 50 }
The text to paste. Useful for special characters that aren’t on the standard keyboard.
Example Request { "action" : "paste_text" , "text" : "Special characters: ©®™€¥£ émojis 🎉" } System Actions
Wait duration in milliseconds.
Example Request { "action" : "wait" , "duration" : 2000 }
No parameters required. Example Request { "action" : "screenshot" }
No parameters required. Example Request { "action" : "cursor_position" }
The application to switch to. Available options: firefox, 1password, thunderbird, vscode, terminal, desktop, directory.
Example Request { "action" : "application" , "application" : "firefox" } Available Applications:
firefox - Mozilla Firefox web browser1password - Password managerthunderbird - Email clientvscode - Visual Studio Code editorterminal - Terminal/console applicationdesktop - Switch to desktopdirectory - File manager/directory browser Response Responses vary based on the action performed:
Default Response Most actions return a simple success response:
Screenshot Response Returns the screenshot as a base64 encoded string:
{ "success" : true , "data" : { "image" : "base64_encoded_image_data" } } Cursor Position Response Returns the current cursor position:
{ "success" : true , "data" : { "x" : 123 , "y" : 456 } } Error Response { "success" : false , "error" : "Error message" } Code Examples curl -X POST http://localhost:9990/computer-use \ -H "Content-Type: application/json" \ -d '{"action": "move_mouse", "coordinates": {"x": 100, "y": 200}}'
Option 1 Option 2 Option 3 Option 4 Option 5 Option 6 Option 7 Option 8 Option 9 Option 10 Option 11 Option 12 Available options:
move_mouse
