Using OmniScout with AI agents

OmniScout is built for AI agents to drive: every command speaks JSON and element refs are stable. Here's how to set it up.

1. Install

pip install omniscout
omniscout install --skill

The --skill flag copies instructions into your agent's skill directory:

The agent picks it up automatically on next session. Other agents (Pi, OpenCode, Windsurf, Cline, Copilot Agent Mode, any shell-capable assistant) can paste the bundled SKILL.md into the system prompt.

2. Get ready

omniscout daemon start

3. Use JSON everywhere

Set OMNISCOUT_JSON=1 at the start of the loop (or pass --json per call). stdout becomes a structured payload; logs go to stderr:

export OMNISCOUT_JSON=1
omniscout search "robotics simulators"
omniscout browser snapshot | jq '.refs[] | select(.role == "button")'

Every response carries enough context (refs, urls, ids) for the next call.

4. Drop a prompt

Research a question

Use OmniScout to research "state of open-source browser agents in 2026" and summarize the top 5 passages with their source URLs.

Navigate and click

Open https://news.ycombinator.com using OmniScout, snapshot interactive elements, click the top story, and screenshot the result to /tmp/hn-top.png.

Log in once and reuse

Run `omniscout browser login https://github.com --profile work`, pause for me to authenticate, then screenshot my notifications page with that profile.

Build a knowledge graph

Build a knowledge graph for "Cursor" with OmniScout. Run `omniscout graph "Cursor" --json --data`, show the tree, and cite sources.

Fill a form

Use OmniScout to open https://example.com, fill the search box with "local-first AI agents", press Enter, wait for results, and list the top 5 results.

Capture network traffic

Open the Vercel pricing page with OmniScout, capture network traffic while scrolling, then list requests matching stripe|payment|checkout.

The agent loop

A reliable loop looks like this:

1. daemon start (idempotent — safe to run every session).
2. export OMNISCOUT_JSON=1.
3. Act: navigate → snapshot --refs-only → click/fill → re-snapshot.
4. Observe: screenshot, get text, network tail, console tail.
5. Clean up: browser close --all.

Isolate parallel tasks with --session <name> so their tabs and refs don't collide.

Debugging agent runs

• omniscout daemon trace — review what the agent did.
• omniscout daemon replay / omniscout replay … — re-run a recorded flow to reproduce a failure.
• omniscout daemon watch — follow along live while the agent works.
• omniscout context — show the current workflow continuity state; omniscout reset to start fresh.

Error kinds

Expect and handle these:

Full list: Troubleshooting.

Tips for agents

• Set OMNISCOUT_JSON=1 at the start for structured output.
• Use omniscout browser snapshot --refs-only to find elements, then click by @eN ref.
• Re-snapshot after every navigate — refs go stale.
• Prefer wait text/networkidle over fixed sleeps.
• Record repeatable flows with record/macro run instead of re-improvising them.
• Close sessions with omniscout browser close --all when done.