Computer commands

omniscout computer controls desktop apps, windows, the clipboard, and the screen on macOS, Windows, and Linux — the desktop counterpart to Browser automation. Commands use the same JSON output and --session model.

Run omniscout computer doctor to check whether desktop control is ready on your machine.

Open, type, keys

omniscout computer navigate /Applications/Notes.app   # app, file, dir, or URL
omniscout computer navigate "Notes" --session notes
omniscout computer type "Meeting notes" --session notes
omniscout computer paste $'First line\nSecond line' --session notes
omniscout computer paste '<b>Meeting notes</b>' --format html --session notes
omniscout computer paste '# Meeting notes' --format md --session notes
omniscout computer key cmd+s
omniscout computer key KP_0
omniscout computer wait 300                            # milliseconds

navigate opens anything the OS can open: app paths/names, files, directories, URLs.
type types into the currently focused window; key sends combos (cmd+s, ctrl+c, cmd+space, F13, KP_0, …).
paste inserts plain text, including multiple lines. On macOS, --format html pastes HTML and --format md converts Markdown to rich HTML when the app supports it. The previous text clipboard content is restored. Linux and Windows currently support plain text only.
ui click accepts --click-count 2 for a double-click or --click-count 3 for a triple-click.
wait sleeps a fixed number of milliseconds — prefer it only for animations; there are no DOM conditions on the desktop.

Clipboard & screenshots

omniscout computer clipboard set "Hello from Scout"
omniscout computer clipboard get
omniscout computer screenshot --out /tmp/desktop.png
omniscout computer screenshot --base64  # include PNG data in JSON output
omniscout computer observe --base64     # accessibility tree and screenshot together
omniscout computer screenshot --region 100,100,800,600 --out /tmp/crop.png  # part of the screen

Use computer paste to insert text without replacing the clipboard. Use clipboard set and clipboard get when you want to move text between the desktop and your shell.

Windows

omniscout computer window list
omniscout computer window activate "Notes"
omniscout computer window activate --id 0x00123456
omniscout computer window resize --id 0x00123456 --width 1200 --height 800
omniscout computer window close --id 0x00123456 --confirm
omniscout computer window close "Notes"

List windows first. Use a window ID to activate, resize, or close that exact window. The selected window receives later input in that session. Closing a window requires --confirm.

UI element interaction

For element-level control instead of blind typing, snapshot the page and act on @eN refs — the same model as browser automation:

omniscout computer ui snapshot
omniscout computer ui click '@e4'
omniscout computer ui click '@e4' --click-count 2
omniscout computer ui drag '@e2' '@e7'
omniscout computer ui is '@e4' button

Full example

# Open Notes and type a message
omniscout computer navigate "Notes" --session notes
omniscout computer wait 300
omniscout computer type "Hello from OmniScout!" --session notes
omniscout computer key cmd+s
omniscout computer screenshot --out /tmp/notes.png --session notes

Cloud equivalent

On OmniScout Cloud, the same primitives run on a remote Sprite via computer.* actions on a unified session — see Endpoints.

See omniscout computer --help (and omniscout computer ui --help) for all options.