Geekflare
Open coreAGPL-3.0Tested by Geekflare

Firecrawl review

Web scraping API that returns Markdown and structured data for AI apps, with a self-hosted edition

Website GitHub Docs
Data refreshedMethodologyHow we tested
Founder of Geekflare

Chandan Kumar is a founder of Geekflare. He is a technology enthusiast and entrepreneur who loves to help businesses and people around the world. Chandan has worked at BNP Paribas, Citibank, Deutsche Bank, Motorola, and HP. He’s got an in-depth understanding of business software and IT infrastructure.

Verdict

Firecrawl's self-hosted API suits teams that already use its SDKs or want the same endpoints on their own servers. Expect a smaller feature set than the cloud and a heavy stack: on my 4 GB test server the default settings ran out of memory, and even trimmed down it used 2.4 GB at idle.

Best for

Teams that use Firecrawl's API or SDKs and need to run it on their own servers

Not for

Teams that need screenshots, page actions or bot handling without paying for the cloud

Pros

  • Same scrape, crawl, map and search endpoints as the cloud API
  • Markdown output, plus LLM extraction with your own OpenAI-compatible model or Ollama
  • SDKs for many languages, and a CLI and MCP server that can point at your own API
  • Very active, with new releases most days

Cons

  • No screenshots or page actions in the self-hosted stack
  • The quickstart turns authentication off, so you must add it before exposing the API
  • The default setup needs more than a 4 GB server, and it builds from source
What people say
Ask Sparkian

Is Firecrawl right for your project?

Describe what you need and AI will check the fit against this review.

Or try:

Firecrawl features and capabilities

Cloud-compatible endpoints

The scrape, crawl, map and search endpoints match Firecrawl Cloud, so code written for the cloud API can point at your own server instead.

Markdown and LLM formats

Pages come back as Markdown or HTML, and JSON extraction works once you connect an OpenAI-compatible model or Ollama.

SDKs, CLI and MCP

Official SDKs for many languages, the Firecrawl CLI and the local MCP server can all target a self-hosted API, so existing integrations keep working.

Background crawl jobs

Crawls run as queued jobs backed by PostgreSQL, Redis and RabbitMQ, so large crawls do not block the API.

JavaScript renderingRenders pages that build their content with JavaScript
YesPlaywright service in the default stack
LLM-ready MarkdownReturns clean Markdown for AI and RAG pipelines
Yes
Structured extractionPulls fields into JSON with selectors or an LLM
YesNeeds your own OpenAI-compatible model or Ollama
Site crawlingFollows links across a site, not just single pages
Yes
Proxy supportRoutes requests through your own proxies
PartlyOne proxy via environment variables; rotation is Cloud only
REST APIAccepts jobs over HTTP so other services can call it
YesAuthentication is off in the quickstart
MCP serverLets AI agents call it as a tool
YesSeparate MCP server that points at your API

At a glance

Type
Scraping API server
Stack
TypeScript / Node.js / Playwright
Databases
PostgreSQL, Redis
Licence terms
The API and engine are AGPL-3.0 and the SDKs are MIT. Screenshots, page actions, the Fire-engine anti-bot service, Agent, Browser and the dashboard are not part of the self-hosted stack.
SSO
No
Roles and permissions
No
Audit logs
No

Free vs paid editions

What you get when you self-host, and what needs a paid plan.

The self-hosted edition covers the scrape, crawl, map and search APIs with plain fetching and Playwright rendering, and LLM extraction once you connect an OpenAI-compatible provider or Ollama. Screenshots, page actions, the advanced anti-bot service, Agent, Browser and the dashboard are not in the self-hosted stack. Firecrawl Cloud has a free plan with 1,000 credits, and paid plans start at $16 a month billed yearly or $19 billed monthly.

Self-hosting Firecrawl

Firecrawl does not publish a minimum host size. Its Compose file runs the API, a Playwright service, Redis, RabbitMQ and PostgreSQL, and caps the API at 4 CPUs and 8 GB of RAM and the Playwright service at 2 CPUs and 4 GB.

New cloud server to ready

11 min 46 s

Memory in use at idle

2411 MB of 4 GB

50-page crawl

1 min 27 s, 2627 MB peak

Version tested

v2.11.538 · Oct 18, 2026

  • Building the stack from source took 11 minutes on 2 CPUs, most of it compiling a Rust component. Then the API refused to start, because the Compose file gives it 4 CPUs and the server has 2.
  • With that fixed, the default five queue workers and browser pool ran the 4 GB server out of memory. The kernel killed the API process and the server stopped answering SSH for about 20 minutes.
  • With one queue worker and lower concurrency it came up in 34 seconds and scraped its first page. At idle the API alone used 1.7 GB, and the whole stack 2.4 GB.
  • The JavaScript test page rendered correctly, and product pages came back as clean Markdown with price, stock and description.
  • The FoundationDB services start by default even though they are only used when you switch the queue backend, so I scaled them to zero. There is no dashboard in the self-hosted version, and API authentication is off unless you set it up.
Commands we ran, with the changes a 2 CPU, 4 GB server needed
git clone https://github.com/firecrawl/firecrawl.git
cd firecrawl && git checkout v2.11.538
cat > .env <<'ENV'
USE_DB_AUTHENTICATION=false
POSTGRES_PASSWORD=change-me
NUQ_WORKER_COUNT=1
NUM_WORKERS_PER_QUEUE=2
MAX_CONCURRENT_JOBS=2
CRAWL_CONCURRENT_REQUESTS=4
ENV
# In docker-compose.yaml: set the api service's "cpus: 4.0" to 2.0,
# publish "127.0.0.1:3002:3002", and add NUQ_WORKER_COUNT to the shared
# environment block so the .env value reaches the container.
docker compose up --build -d --scale foundationdb=0 --scale foundationdb-init=0
curl -X POST http://127.0.0.1:3002/v2/scrape -H 'Content-Type: application/json' \
  -d '{"url": "https://books.toscrape.com/", "formats": ["markdown"]}'

What you can do with Firecrawl

Drop-in for Firecrawl Cloud

Point existing code that uses Firecrawl's SDKs at your own server to keep scraped pages in your own infrastructure.

Markdown for RAG

Crawl a documentation site or knowledge base and feed the Markdown into a vector database for a support chatbot.

Site maps for audits

The map endpoint lists every URL on a site quickly, which helps content and SEO audits before a full crawl.

Firecrawl alternatives

Crawl4AIOpen source crawler that turns web pages into Markdown and structured data for LLMsScrapyPython framework for writing fast, large-scale web crawlers and scrapersCrawleeNode.js library for building crawlers that use plain HTTP or a headless browserMaxunNo-code web scraping platform that turns websites into APIs and spreadsheetsScraplingPython scraping library with fast HTTP requests, browser fetching, Scrapy-style spiders and Markdown outputPlaywrightBrowser automation library from Microsoft for Chromium, Firefox and WebKit