Firecrawl is the easiest way to extract data from the web. Developers use us to reliably convert URLs into LLM-ready markdown or structured data with a single API call.
This is the 6th launch from Firecrawl. View more
Firecrawl CLI
Launched this week
Firecrawl CLI is an all-in-one toolkit for scraping, searching, and browsing the web. Built for AI agents and developers, it delivers clean, reliable data with maximum token efficiency - outperforming native Claude Code fetch with >80% coverage.
Free Options
Launch Team
@ericciarla interesting launch.
Reading through the CLI and the way Firecrawl turns web pages into clean markdown or structured data for agents, something stood out.
It seems to behave less like a traditional scraping tool and more like a data ingestion layer for AI agents.
Especially when agents can search, crawl and structure web data as part of their reasoning workflows.
Curious how you see Firecrawl evolving internally.
Is it mainly a scraping toolkit, or closer to infrastructure for real-time web data pipelines in agent systems?
Firecrawl
Hey Product Hunt! 👋 Eric here.
We launched Firecrawl CLI, your agent’s complete web data toolkit.
Every developer building with agents eventually hits the same wall, reliable web data access.
Most tools break on JavaScript-heavy sites or dump entire pages into context, wasting tokens and slowing down reasoning.
Firecrawl CLI fixes that.
It gives agents a unified interface to:
- scrape pages into clean Markdown or JSON
- search and return complete results in one step
- browse interactive or gated pages through a cloud browser
- crawl and map entire sites for structured coverage
Firecrawl CLI uses a file-based approach for context management, so results are written to the filesystem and agents can use bash methods for efficient search and retrieval.
To install it, just run:
$ npx -y -cli@latest init --all --browser
Once installed, your agent knows how to get live web data whenever it needs it.
Real use cases:
- Enrich AI agents and knowledge bases with live, structured web data for more accurate reasoning and responses
- Power deep research workflows by collecting verifiable sources, documentation, and papers across the web
- Automate data and market intelligence gathering - tracking competitors, product updates, and industry trends in real time
- Capture information from dynamic or login-gated sites to access data hidden behind dashboards, forms, and authenticated workflows.
Works with all popular harnesses like Claude Code, Codex, and OpenCode.
You can try it now: https://docs.firecrawl.dev/sdks/cli
We’d love to hear what you build with it!
the file-based approach for context management is such a smart design choice. been building agents that need web data and the biggest headache is always getting clean output without burning tokens on garbage html. curious how the cloud browser handles sites with heavy client-side rendering tho, like SPAs built on next.js or similar?
Firecrawl
@mihir_kanzariya Thank you! It handles heavy client side sites very well - that is the exact reason we built it to complement and extend scrape :)
Firecrawl
@bengeekly We do!!
AutonomyAI
Awesome project! shared with our dev team :)
Firecrawl
@lev_kerzhner Sweet - ty!!
Reliable web data access is definitely one of the biggest pain points when building agents. The idea of returning clean structured data instead of dumping full pages into context makes a lot of sense. How does Firecrawl CLI handle sites that actively block scraping or frequently change their structure?
MacQuit
Really cool to see this evolve into a full CLI toolkit! As a developer who works with web data for building AI features, the token efficiency angle is super practical — wasted tokens on messy HTML is a real pain point. Love that it's open source too. Congrats on the launch!