Browser

Browser automation for navigation, interaction, and screenshots

What it does: Drives a headless Chromium browser via Playwright — navigate, click, type, screenshot, read pages via accessibility snapshots, manage tabs and isolated profiles, and run JavaScript.

Who it’s for: Anyone whose agent needs to interact with sites that don’t yield to a simple web_fetch — single-page apps, sites behind logins, dynamic dashboards, file uploads, or workflows that require multiple steps. For static page content, the lighter web_fetch tool is faster and cheaper.

Browser Actions

The browser tool supports 16 actions, verified against BROWSER_TOOL_ACTIONS in packages/skills/src/builtin/platform/browser-tool-schema.ts:

Action	What It Does
status	Check if the browser is running or stopped
start	Launch the browser
stop	Close the browser
profiles	Manage isolated browser profiles
tabs	List all open tabs
open	Open a new tab with a URL
focus	Switch to a specific tab (by targetId)
close	Close a tab (by targetId)
snapshot	Get the page accessibility tree (aria or ai format)
screenshot	Capture a page screenshot
navigate	Go to a URL in the current tab
console	View browser console output
pdf	Save the current page as a PDF
upload	Upload a file to the page
dialog	Handle browser dialogs (alerts, confirms, prompts)
act	Interact with page elements (11 sub-kinds)

Page Actions

navigate -- Go to a URL

Navigate the current tab to a new URL.

Parameter	Type	Required	Description
action	string	Yes	"navigate"
targetUrl	string	No	The URL to navigate to

The browser loads the page and waits for it to finish loading before returning. The request guard screens the destination before navigation (see Security below).

screenshot -- Capture a screenshot

Take a screenshot of the current page. The image is returned as a message attachment.

Parameter	Type	Required	Description
action	string	Yes	"screenshot"
targetId	string	No	The tab to capture (defaults to the active tab)
fullPage	boolean	No	Capture the full scrollable page (not just the viewport)
type	string	No	Image format: "png" (lossless) or "jpeg" (compressed)

Screenshots are useful for visual verification — checking how a page looks, confirming form submissions, or capturing error states.

snapshot -- Get the accessibility tree

Get a structured representation of the page content using the accessibility tree. This is how agents “read” web pages.

Parameter	Type	Required	Description
action	string	Yes	"snapshot"
targetId	string	No	The tab to snapshot (defaults to the active tab)
snapshotFormat	string	No	"aria" for the raw accessibility tree, or "ai" for an AI-optimized format
mode	string	No	Snapshot mode: "efficient" for compact output optimized for LLM consumption
refs	string	No	Element reference type: "role" (ARIA role references) or "aria" (ARIA label references)
interactive	boolean	No	Show only interactive elements
compact	boolean	No	Use compact output format
depth	number	No	Maximum tree depth to include
maxChars	number	No	Maximum characters in the snapshot output

Format options:

• aria — The full accessibility tree with all ARIA roles and properties. Detailed but verbose.
• ai — A condensed format designed for AI consumption. Shows interactive elements (buttons, links, inputs) with labels and references that can be used with the act action.

The ai format is recommended for most use cases — it gives agents the information they need to interact with the page without overwhelming them with structural details.

act -- Interact with page elements

Interact with elements on the page. The act action supports 11 different interaction types (called “sub-kinds”), each designed for a specific type of interaction.

Parameter	Type	Required	Description
action	string	Yes	"act"
request.kind	string	Yes	The interaction type (see table below)
request.*	(varies)	(varies)	Additional parameters depend on the kind

The act action wraps its interaction parameters inside a request object. See the Interaction Types section below for all 11 sub-kinds and their parameters.

open -- Open a new tab

Open a new browser tab and navigate to a URL.

Parameter	Type	Required	Description
action	string	Yes	"open"
targetUrl	string	No	The URL to open in the new tab

The same request guard used by navigate applies before the new tab loads its URL.

pdf -- Save page as PDF

Save the current page as a PDF document, returned as a message attachment.

Parameter	Type	Required	Description
action	string	Yes	"pdf"

Useful for archiving web pages or generating printable versions of online content.

upload -- Upload files

Upload one or more files to a file input element on the page.

Parameter	Type	Required	Description
action	string	Yes	"upload"
inputRef	string	No	CSS selector or aria reference for the file input element
paths	string[]	No	Array of file paths to upload

dialog -- Handle browser dialogs

Respond to browser dialogs such as JavaScript alerts, confirmation prompts, and input prompts.

Parameter	Type	Required	Description
action	string	Yes	"dialog"
accept	boolean	No	Whether to accept (true) or dismiss (false) the dialog
promptText	string	No	Text to enter in a prompt dialog

console -- View console output

Retrieve the browser’s developer console output, including errors, warnings, and log messages.

Parameter	Type	Required	Description
action	string	Yes	"console"

Useful for debugging page issues or monitoring JavaScript errors.

Interaction Types (act sub-kinds)

When using the act action, the kind parameter determines what type of interaction to perform. There are 11 interaction types:

Kind	What It Does	Key Parameters
click	Click an element	ref (element reference or CSS selector), doubleClick, button, modifiers
type	Type text character by character	ref, text, submit, slowly
press	Press a keyboard key	key (e.g., "Enter", "Tab", "Escape")
hover	Hover over an element	ref
drag	Drag from one element to another	startRef, endRef (both element references)
select	Select a dropdown option	ref, values (array of option values)
fill	Fill form fields (clears existing content first)	ref, fields (array of field objects)
resize	Resize the browser viewport	width, height
wait	Wait for a condition	timeMs (milliseconds), textGone (text to wait to disappear)
evaluate	Run JavaScript in the page	fn (JavaScript code)
close	Close the current page or dialog	(no additional parameters)

Selecting Elements

Most interaction types require a ref parameter to identify which element to interact with. You can use:

• CSS selectors — Standard CSS selectors like #login-button, .submit-btn, or input[name="email"]
• Aria references — References from the snapshot output in ai format, which label interactive elements with short identifiers

The recommended workflow is: take a snapshot in ai format to see the page structure, then use the element references from the snapshot in your act calls via the ref parameter.

Browser Profiles

Profiles create isolated browser contexts with separate cookies, local storage, and session data. This is useful for:

• Managing multiple accounts on the same website
• Testing with different user states (logged in vs. logged out)
• Keeping browsing sessions separate between different tasks

Use the profiles action to create, list, and switch between browser profiles.

Browser runtime options

The Linux installer provisions stock Chrome running headed via Xvfb when --with-browser and --with-xvfb are enabled (both are installer defaults). Use --with-cloakbrowser to opt into the alternative browser runtime. A --without-xvfb install runs Chrome headless; --without-browser skips the browser runtime entirely. No runtime guarantees access to sites with bot detection, authentication challenges, or IP-reputation controls.

The install-time flags compose. They’re available in both the bare-metal installer (install.sh) and the Docker image (build args):

Flag	What it adds	Default
--with-browser	Stock Google Chrome + headless shared libs. Sandbox ReadWritePaths widened for Chrome’s out-of-profile writes (~/.config/google-chrome, ~/.local/share/applications).	On (opt out with --without-browser). The browser tool needs a Chrome binary to launch.
--with-xvfb	Adds the system-scope comis-xvfb.service companion on a managed Linux install. Both units bind the shared /run/comis-x11 socket directory into their private /tmp; config is seeded with headless: false only when the companion is available. User-scoped systemd installs downshift to headless mode.	On for the managed system service (opt out with --without-xvfb).
--with-cloakbrowser	Installs the optional CloakBrowser Chromium runtime. findChrome() probes it before the stock Chrome paths.	Opt-in. Compatibility and detection outcomes vary by site and network.

Sites may block automated browsers or datacenter networks independently of the browser fingerprint. CloakBrowser does not include a proxy and does not bypass site access controls. Follow each site’s terms and automation policies.

Install examples

# Managed Linux host: download, inspect, preview, then run the selected option
curl -fsSL --proto '=https' --tlsv1.2 https://comis.ai/install.sh -o comis-install.sh
less comis-install.sh
bash comis-install.sh --dry-run --with-cloakbrowser --with-xvfb
bash comis-install.sh --with-cloakbrowser --with-xvfb

# Docker: same matrix via build args
docker build \
  --build-arg COMIS_WITH_CLOAKBROWSER=1 \
  --build-arg COMIS_WITH_XVFB=1 \
  -t comisai/comis:cloak-xvfb .

# Docker: exercise the installer's non-interactive package/browser branches
docker build -f Dockerfile.install \
  --build-arg COMIS_WITH_CLOAKBROWSER=1 \
  --build-arg COMIS_WITH_XVFB=1 \
  -t comis-installed:cloak-xvfb .

Comis probes the configured and supported browser paths at runtime. Confirm the selected binary and launch mode with comis doctor and the daemon logs after installation.

License note

CloakBrowser’s compiled binary has a separate license. Review the current CloakBrowser binary license before using or redistributing it.

Security

The browser includes built-in security protections:

• HTTP(S) request validation — The requested destination, every HTTP(S) redirect hop, subresources, and page fetches in guarded tabs are checked against localhost, private-network, link-local, reserved, and cloud-metadata address ranges before Chromium continues them.
• Fail-closed new tabs — Tabs opened with the browser tool are guarded before navigation. A page-created popup whose first request arrives before a page guard exists is blocked; use the browser open action when a workflow needs a new tab.
• Connected-page service-worker bypass — Each connected page bypasses an existing service worker before request interception is enabled. Chromium does not disable workers for the whole persistent profile, so workers outside a connected page remain outside this page-level control.
• Unintercepted transport lockdown — Dedicated and shared workers, WebTransport, WebSocketStream, and WebRTC are disabled in guarded page contexts because their traffic cannot be validated by the page request interceptor. New service-worker registrations are rejected. Sites that require these APIs will have reduced functionality.
• Window WebSocket validation — WebSocket connections created by guarded page scripts receive the same hostname and IP-range checks before Playwright connects to the server. If a document was already loaded when Comis attached its guard, WebSocket remains disabled in that document until the next navigation.
• Network-isolation boundary — Chromium performs its own DNS resolution after validation, so browser traffic is not connection-level DNS-pinned. A persistent profile can also contain browser-managed background state outside the connected page. These controls are not a host firewall: keep the browser isolated from sensitive internal networks and use web_fetch for static retrieval that does not require browser interaction.
• Screenshot sanitization — The standard daemon validates screenshot dimensions and format, and resizes or re-encodes captures when required. If the configured sanitizer rejects a capture, the action fails and does not return the original image bytes.

For more details on network security protections, see Security.

End-to-end example: scraping a SPA

A typical multi-step workflow — visit a JavaScript-rendered dashboard, log in, navigate to a data view, and pull a value the agent can act on. The agent strings together navigate, snapshot, act, and screenshot in sequence:

# 1. Make sure the browser is running
action: status
# (start it if not)
action: start

# 2. Open the target URL in a new tab
action: open
targetUrl: "https://app.example.com/login"

# 3. Read the page structure to find the login form
action: snapshot
snapshotFormat: ai
interactive: true

# 4. Fill credentials using the refs from the snapshot
action: act
request:
  kind: fill
  fields:
    - { ref: "email-input", value: "[e-mail]" }
    - { ref: "password-input", value: "${LOGIN_PASSWORD}" }
# (the value substitution happens at the agent layer; the secret store
# resolves ${LOGIN_PASSWORD} to the encrypted value)

# 5. Submit the form
action: act
request:
  kind: click
  ref: "submit-button"

# 6. Wait for the dashboard to render after login
action: act
request:
  kind: wait
  textGone: "Signing in..."

# 7. Navigate to the data view
action: navigate
targetUrl: "https://app.example.com/dashboard/metrics"

# 8. Read the rendered DOM via accessibility tree
action: snapshot
snapshotFormat: ai
maxChars: 5000

# 9. Capture a screenshot for the response
action: screenshot
fullPage: true

The agent now has the rendered text in a structured snapshot and a visual screenshot, and can post both back to the user via the message tool’s attach action.

For sites that emit network calls you want to inspect, the agent can pair the browser with console (to see JS logs) and act kind: evaluate (to run JavaScript like document.querySelector(...).innerText). For high-throughput scraping that does not need rendered JavaScript, prefer web_fetch.

Related

Built-in Tools

All built-in tools including browser

Web Tools

Web search and page fetching tools

Agent Tools Overview

See all available agent tools

Config Reference

Browser and tool configuration options