What it is
Crawl4AI crawls web pages and turns them into Markdown for language models, RAG pipelines and datasets. It drives a real Chromium through Playwright, so JavaScript-heavy pages render, and it can extract structured data with CSS, XPath or an LLM, crawl many URLs at once under a memory-aware dispatcher, and run in a stealth mode.
It comes three ways: a Python library, a self-hosted Docker server with a dashboard and MCP endpoint, and Crawl4AI Cloud, a hosted API with /scrape, /extract, /search, /serp and /answer. The cloud gives 1,000 free credits a month, with paid plans from $10.
The code is Apache 2.0, but the LICENSE file appends an attribution requirement: public uses and distributions must credit UncleCode and the Crawl4AI project.
Our review
Crawl4AI remains the open-source default for turning web pages into Markdown, with about 86,100 GitHub stars and a steady cadence of six releases from v0.9.0 on 18 June to v0.9.5 on 14 October 2026. Two things have changed since earlier coverage: the project now sells a hosted API, Crawl4AI Cloud, which the README promotes heavily, and the Docker server has become its weak point.
The security record is the first thing a self-hoster should read. GitHub lists 25 advisories published in 2026, five rated critical, almost all against the Docker server: pre-authentication remote code execution through a sandbox escape, RCE via Chromium launch-argument injection, path-traversal file writes and a run of server-side request forgery bypasses. The October batch alone added six, including Chromium launched with TLS verification switched off. The library used inside your own Python code is a smaller attack surface; the server belongs on a private network, behind a token, on the latest release.
The licence needs a second look too. The repository says Apache 2.0, yet the LICENSE file adds that public uses, including websites and command-line tools, must credit UncleCode and Crawl4AI in an About page, NOTICE file or help output. That is lighter than Firecrawl's AGPL, but legal teams that clear Apache 2.0 by default will want to see it.
Output quality is good but still takes tuning. Open issues report Markdown that loses heading hierarchy and tables, browser processes accumulating for weeks on the Docker /crawl path, and a memory leak on repeated /md calls that crashes containers. Anti-bot work is yours when self-hosted: stealth mode and an undetected-browser adapter help, but you buy the proxies. The cloud handles bot walls, charging ISP proxies at 2x and residential proxies at 4x the base credit cost.
Against Firecrawl the trade is now clearer. Both have an open core and a hosted API with 1,000 free credits; Firecrawl's cloud is older, publishes full prices and claims SOC 2 Type II, while Crawl4AI's is a soft launch whose per-endpoint prices load live and may change. For self-hosting, Crawl4AI's library is far easier to drop into one Python job than Firecrawl's multi-service stack.
Where it earns its place
A data engineer refreshing a RAG index of 5,000 documentation pages nightly
Re-crawl vendor docs and push clean Markdown into a vector store
Call arun_many from a scheduled Python job with the memory-adaptive dispatcher; there are no per-page fees, only the VM. Pin the version and re-check the advisory list at each upgrade.
A platform team offering web access to internal agents
Run one shared crawl service that agents reach over MCP
Run the Docker image on port 11235 with --shm-size=1g on a private subnet behind a token, and point agents at its MCP endpoint. Budget time for patching: 25 advisories landed in 2026, mostly on this server.
A two-person startup with nobody to manage proxies
Scrape bot-protected retail sites for a price tracker
Crawl4AI Cloud's Starter plan at $10 a month gives 10,000 credits, 120 requests a minute and 10 at a time; residential proxies cost 4x the base rate. The prices are labelled a soft launch, so avoid long commitments.
Before you commit
- Setup
- pip install -U crawl4ai, then crawl4ai-setup to install Chromium; Python 3.10 or later. The Docker server is one docker run with port 11235 and --shm-size=1g, and includes a dashboard and playground. A first page to Markdown takes minutes.
- Integrations
- Python async API (AsyncWebCrawler, arun_many), a REST Docker server with MCP, and the cloud's /scrape, /extract, /search, /serp and /answer endpoints with a hosted MCP URL for Claude Code, Codex, Cursor and OpenCode. LLM extraction uses any provider key you supply.
- Data and privacy
- Self-hosted, pages go only where you send them, and the README mentions no telemetry. The cloud keeps request parameters and result summaries for 90 days, caches fetched pages, and lets you pin EU, Singapore or US regions. The LICENSE requires attribution in public uses.
- Cost at scale
- Self-hosting costs the VM and any proxies; no official sizing is published. Cloud Starter is $10 a month for 10,000 credits; extra credits cost $10 to $1,000 at the plan's rate and never expire, and 5xx failures are not charged.
- Limits
- 278 open issues and pull requests; reported memory leaks and accumulating browser processes on the Docker server; Markdown can lose table structure; the Docker API lags library features, with an open issue tracking parity.
What's good
- Apache 2.0 code, embeddable as a Python library
- No per-page cost when self-hosted
- Memory-adaptive concurrent crawling and stealth mode
- Hosted cloud with MCP, from $10 a month
What's not
- 25 security advisories in 2026, five critical, mostly the Docker server
- LICENSE adds a mandatory attribution clause
- Proxies and anti-bot work are yours when self-hosted
- Cloud prices are a soft launch and may change
Best for
Python teams that want a self-hosted crawler for RAG or agents without per-page costs.
Alternatives
Crawl4AI: common questions
- Is Crawl4AI worth it?
- Still the strongest open crawler for LLM pipelines, but its Docker server's 2026 security record rules out exposing it without hardening. We rate it 4.2 out of 5.
- How much does Crawl4AI cost?
- Library and Docker server free (Apache 2.0 plus an attribution clause). Crawl4AI Cloud: 1,000 free credits a month; Starter $10 a month for 10,000 credits.
- What are the downsides of Crawl4AI?
- 25 security advisories in 2026, five critical, mostly the Docker server; lICENSE adds a mandatory attribution clause; proxies and anti-bot work are yours when self-hosted; cloud prices are a soft launch and may change.
- Who is Crawl4AI best for?
- Python teams that want a self-hosted crawler for RAG or agents without per-page costs.
- How much does Crawl4AI cost for a team or in production?
- Self-hosting costs the VM and any proxies; no official sizing is published. Cloud Starter is $10 a month for 10,000 credits; extra credits cost $10 to $1,000 at the plan's rate and never expire, and 5xx failures are not charged.
- Is Crawl4AI safe for company data?
- Self-hosted, pages go only where you send them, and the README mentions no telemetry. The cloud keeps request parameters and result summaries for 90 days, caches fetched pages, and lets you pin EU, Singapore or US regions. The LICENSE requires attribution in public uses.
- What are the best Crawl4AI alternatives?
- The closest alternatives we have reviewed are Firecrawl and Browser Use.
Answers checked Oct 23, 2026.
Make Crawl4AI? Show this rating on your site
