v0.17.0New: acrawl run replays saved crawls with zero LLM calls

The open-source AI browser agent Describe it. acrawl it.

An open-source browser agent and MCP server in a single Rust binary. Tell it what you want from the web in plain English. It navigates, clicks, fills forms, watches the network, and hands back structured data.

curl -fsSL https://raw.githubusercontent.com/Mingye-Lu/AgenticCrawler/main/install.sh | bash

irm https://raw.githubusercontent.com/Mingye-Lu/AgenticCrawler/main/install.ps1 | iex

git clone https://github.com/Mingye-Lu/AgenticCrawler.git && cd AgenticCrawler && cargo build --release

Verifies the release checksum and sets up the stealth browser. Browser features need Node.js 20+.

Run your first crawl Read the docs

~/crawls · acrawl

01 You describe the goal.
02 The agent plans, browses and extracts.
03 It saves the repeatable part as a script.
04 acrawl run replays it. No model, no key, no cost.

43agent tools

40MCP tools

25LLM providers

17MCP clients

0LLM calls per script step

1binary

Bring your own model

Anthropic
OpenAI
Google Gemini
DeepSeek
Amazon Bedrock
Azure OpenAI
Vertex AI
GitHub Copilot
GitLab Duo
Groq
Cerebras
Mistral AI
xAI
Cohere
Alibaba
OpenRouter
Together AI
Ollama

25 providers, plus any OpenAI-compatible endpoint. See them all →

AI once, then a script

Stop paying a model to click “next page”.

A ReAct-style agent calls the model on every page, every run. acrawl’s agent works the job out once, saves the repetitive part as a JSON script with loops, conditionals and parallel branches, and replays it from MCP, the REPL or acrawl run, straight to tools.

Pages per run 50Runs 30

Agent on every page

1,500

acrawl script

~1 run’s worth

Model calls, at one per page. The script’s only model cost is the first run that writes it.

// books_catalog.json: written once by the agent
{
  "schema_version": 1,
  "name": "books_catalog",
  "steps": [{
    "type": "for_each", "variable": "u",
    "iterable": { "kind": "variable", "value": "urls" },
    "steps": [
      { "type": "tool_call", "tool": "navigate",
        "input": { "url": "$u" } },
      { "type": "tool_call", "tool": "execute_js",
        "input": { "script": "…" }, "output": "rows" },
      { "type": "collect",
        "value": { "kind": "variable", "value": "rows" } }
    ]
  }]
}

$ acrawl run books_catalog --format jsonl -o books.jsonl
script completed: 40 item(s), 13 step(s), 6.2s

$ crontab -l
  0 7 * * *  acrawl run books_catalog -q -o /data/books.json

No provider credentials needed to replay
Validated before the browser launches; bad input exits 2
Partial data still written on failure or Ctrl+C

How scripts work →

Acts and observes

Your agent gets DevTools, not just a mouse.

Most browser agents see pixels or HTML. acrawl’s agent, and any MCP client, can ask the page what it actually did: which requests it made, which errors it threw, how fast it loaded and which WCAG rules it breaks.

list_network_activity()

Method	URL	Status	Type	Time
GET	/api/search?q=laptop	200	xhr	184 ms
GET	/static/app.9f2c.js	200	script	62 ms
POST	/api/cart/add	409	fetch	231 ms
GET	/img/hero@2x.webp	200	image	96 ms
GET	wss://live.example.com/feed	101	websocket	open

list_page_logs()

Level	Message	Source	Count
error	TypeError: price is undefined	checkout.js:212	×14
warn	Deprecated API: document.domain	vendor.js:3	×1
log	cart hydrated (3 items)	app.js:88	×1
error	Failed to load /api/reviews (503)	reviews.js:40	×2

Metric	Value
TTFB	142 ms
DOMContentLoaded	1.18 s
Load	2.64 s
Transfer	2.1 MB over 86 requests
Largest resource	app.9f2c.js · 712 KB

audit_accessibility()

Impact	Rule	Affected	Standard
serious	color-contrast	7 nodes	WCAG 1.4.3
critical	button-name	2 nodes	WCAG 4.1.2
moderate	heading-order	1 node	best practice
minor	image-redundant-alt	3 nodes	best practice

Also:inspect_requestinspect_websocketinspect_cookiesinspect_storagemeasure_coverageintercept_networkinspect_loglist_resources

Debug a live page with acrawl →

Under the hood

Built to crawl, not to demo.

[01] one binary

No Python. No Docker.

acrawl is a single compiled Rust executable. A bundled stealth browser handles JavaScript pages through a small Node.js bridge.

[02] smart fetch

HTTP first, browser when needed

Static pages come over plain HTTP. Framework markers, auth redirects or an empty SPA shell switch the request to a headless browser automatically.

[03] sub-agents

Fork onto parallel tabs

The agent can split work across child agents, each with its own tab and step budget. A URL-claim registry keeps siblings off the same page.

[04] any model

25 LLM providers

Anthropic, OpenAI, Gemini, DeepSeek, Bedrock, Azure, Vertex, Copilot, Groq, Mistral, xAI, OpenRouter and more, plus any OpenAI-compatible endpoint, local models included.

[05] your browser

Logged-in sessions

The acrawl Bridge extension lets the agent drive your real Chrome, Edge or Brave, with your cookies and sign-ins, for sites a headless browser can’t reach.

[06] token diet

Pages trimmed to what matters

fit_markdown strips navigation and boilerplate, typically saving 30–60% of tokens. An optional focus prunes pages down to the blocks your goal is about.

MCP both ways

A real browser for the agent you already use.

acrawl mcp gives your coding agent 40 tools over stdio: 32 browser and DevTools tools, 7 script tools and run_goal, which hands a whole task to acrawl’s own agent. Onlyrun_goal needs an LLM key.

$ acrawl mcp install
  detected: Claude Code, Cursor, VS Code
  ✓ wrote 3 configs

Set up the MCP server →

Promo · 52 seconds

acrawl in 52 seconds.

Honest comparison

How acrawl stacks up.

	acrawl	browser-use	Playwright MCP	Firecrawl
Plain-English goal, own agent loop	✓	✓	—	hosted
Runs fully on your machine	✓	✓	✓	self-host option
DevTools observation as tools	full	—	console + network	—
Deterministic scripts, zero LLM calls	✓	—	—	—
Install	one binary	Python package	Node package	API / self-host
License	MIT	MIT	Apache 2.0	AGPL-3.0

Reviewed October 2026 against each project’s own README. Spot something out of date?Open an issue.Full comparison →

FAQ

Questions people ask.

What is acrawl?

acrawl (the project is called AgenticCrawler) is an open-source, MIT-licensed browser agent and MCP server written in Rust. You describe a web task in plain English and it navigates, interacts and returns structured data. It also exposes DevTools-style observation (network, console, performance, cookies, storage, coverage, accessibility) as agent tools.

Do I need to write code?

No. Goals are plain English, for example acrawl prompt "scrape all book titles and prices from books.toscrape.com". For repeatable, cheaper runs you can save the repetitive part as a JSON script and replay it with acrawl run.

Which LLMs does it work with?

25 providers, including Anthropic, OpenAI, Google Gemini, DeepSeek, AWS Bedrock, Azure OpenAI, Vertex AI, GitHub Copilot, Groq, Mistral, xAI, Cohere, Alibaba DashScope and OpenRouter, plus any OpenAI-compatible endpoint, including local models.

Can I use it from Claude Code, Cursor or VS Code?

Yes. Run acrawl mcp install and pick your clients. The MCP server exposes 40 tools: 32 browser and DevTools tools, 7 script tools and run_goal. Only run_goal needs an LLM key; the rest work without one.

Can it handle JavaScript-heavy sites and SPAs?

Yes. acrawl fetches over HTTP first and escalates to a headless stealth browser when it detects framework markers, auth redirects or an empty SPA shell.

Can it log in to sites or get past CAPTCHAs?

It can fill login forms and type a 2FA code you paste in, but it can't receive or generate codes, and it won't solve image or proof-of-work CAPTCHAs. For logged-in or heavily protected sites, use your real browser through the acrawl Bridge extension.

How does it keep costs down?

Static pages skip the browser entirely. fit_markdown strips boilerplate, typically saving 30–60% of tokens. Deterministic scripts run repetitive steps with no LLM calls, and sub-agents work in parallel with their own step budgets.

All questions →

Point it at the web.

Free, open source, MIT-licensed. One command to install.

curl -fsSL https://raw.githubusercontent.com/Mingye-Lu/AgenticCrawler/main/install.sh | bash

Install guide Star on GitHub · 15