Quick start
This page takes you from a fresh install to a finished crawl. You need an API key for one LLM provider, or a local model server such as Ollama.
1. Connect an LLM provider
Section titled “1. Connect an LLM provider”acrawl auth anthropic # or: openai, google, other, ... (acrawl auth list shows all 25)acrawl asks for your key and stores it in ~/.acrawl/credentials.json. Set ACRAWL_CONFIG_HOME to use a different config directory. For a local model, run acrawl auth other --base-url http://localhost:11434/v1. LLM providers covers every provider and how to pick a model.
Check it worked:
acrawl auth status2. Run a crawl
Section titled “2. Run a crawl”Start the interactive terminal UI:
acrawlOr run a single goal and exit, which suits scripts and CI:
acrawl prompt "scrape all book titles and prices from books.toscrape.com"The terminal UI streams the agent’s reasoning and every tool call as it happens. Type a goal and press Enter; press Esc twice to interrupt. Sessions save automatically when you quit with /exit.
3. Watch it, steer it, check the cost
Section titled “3. Watch it, steer it, check the cost”- See the browser. Start with
acrawl --headed, or type/headed, to watch the agent click through the page in a visible window. - Steer it. If the agent heads the wrong way, interrupt and add a message such as “only the first page, please”. It keeps the context it has built so far.
- Check the cost.
/costshows tokens and spend for the session;/statusshows the model and context use. - Switch model.
/modelopens a searchable list of every configured provider’s models.
4. Get the data out
Section titled “4. Get the data out”In the terminal UI the agent replies with the data, normally as JSON or a table. For scripts and pipelines, ask for JSON output:
acrawl --output-format json prompt "list the 10 newest posts on example.com/blog with title and date" > result.jsonThe JSON holds the agent’s final message, the model, the tool_uses it made and token usage. Files the agent downloads with save_file go to the output folder, which you can change with acrawl config set output_dir <path>.
Write goals that work
Section titled “Write goals that work”| Instead of | Try |
|---|---|
| “get the products” | “get the name, price and rating of every product on example.com/shop, following pagination” |
| “check the site” | “open example.com, list any console errors and any network requests that failed” |
| “log in and get my orders” | Use your own browser, where you’re already signed in, then “list my last 5 orders” |
Name the site, the fields you want and when to stop. The agent does better with a clear finish line than with an open-ended “everything”.
Example goals
Section titled “Example goals”Scrape a product catalog
acrawl > scrape all book titles, prices, and ratings from books.toscrape.comThe agent reads the page, extracts the data, follows pagination through all 50 pages and returns JSON.
Fill and submit a form
acrawl > go to example.com/contact, fill in name "Jane Doe", email "jane@example.com", message "Hello", and submit the formExtract from a JavaScript-rendered page
acrawl > get all repository names and star counts from github.com/trendingPlain HTTP won’t see this content. acrawl detects the framework markers and escalates to a headless browser on its own.
Crawl in parallel
acrawl > scrape the title, author, and price of every book across all 50 pages on books.toscrape.com. Fork a sub-agent for each page to speed this up.The agent runs up to 5 sub-agents at once, each in its own tab, and merges their results.