crw
fastCRW is a web data engine for AI agents, written in Rust and run as a small binary or through a managed API. It offers five operations: Scrape (one URL to markdown, HTML, links, screenshots or schema JSON), Crawl (a bounded crawl of a site), Map (discover URLs without scraping every page), Search (web search with optional scraping of results) and Extract (structured fields from one or many URLs). The README says crawl and map follow `robots.txt` by default and asks users to respect website policies.
Install with `curl -fsSL https://fastcrw.com/install | sh`, which runs locally for free with no account. If you pass `CRW_API_KEY`, the same installer connects your cloud key and registers the MCP server with tools it detects, such as Claude Code, Cursor, Codex, Gemini CLI, OpenCode and Windsurf; set `CRW_NO_AGENTS=1` to skip that step. The managed API adds proxies, JS rendering and search, with 1,000 free credits advertised, while self-hosting leaves renderers, search, auth, proxies and capacity to you.
The README makes benchmark claims: on Firecrawl's public 1,000-URL dataset it says fastCRW recovered more ground truth than Crawl4AI and Firecrawl with about 14 MB idle RAM, and that it answers 90.0% of 600 AA-Omniscience questions correctly. Both are the vendor's own results, with methodology linked. Licensing is split: the engine and MCP server are AGPL-3.0 and the Python and TypeScript SDKs are MIT, with a separate embedding licence available. Last push was 2026-10-13. This review is based on the README and repository data; we have not run it ourselves.
Features
Why choose it
Trade-offs
Maintenance check
- Last commit
- 2026-11-06
- Commits, last 90 days
- 100+
- Releases, last 90 days
- 18
- Latest release
- v0.42.1 (2026-11-06)
- Open issues
- 7
- Recent issues with at least one comment
- 10/10
Oldest week on the left; the latest four weeks are orange. Source: GitHub commit activity.
Status rule: archived on GitHub = Archived; last commit within 90 days = Active; 91 to 365 days = Slowing; older = Stale. All figures come from the GitHub API on the date shown.
Most discussed open issues
- #660 `onlyMainContent` (default true) still emits filter UI, cart drawer, live-chat widget and cookie/browser-warning text5 comments · 2026-11-01
- #658 Infinite-list / "Load more" collection pages return only the first page of results; `?page=N` returns identical content and there is no way 2 comments · 2026-11-01
- #637 camoufox tier: reuse Cloudflare clearance across requests (per-host session) and allow camoufox-first ordering1 comments · 2026-10-16
- #590 Add crw to awesome-ai-plugins?0 comments · 2026-09-29
- #586 Proposal: optional fee-less per-request Nano (XNO) settlement rail beside credit billing0 comments · 2026-09-25
Issue titles are quoted from GitHub. Open the link to check whether a problem still applies to the version you use.
Compatibility
Quick start
Use cases
Alternatives
Comments
No comments yet. Be the first!