Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Repository Index
110,088 visible repositories in the index
1–5 of 5 · Most stars
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
A Rust library to extract useful data from HTML documents, suitable for web scraping.
Local-first web & document ingestion engine in Rust — HTTP/browser capture, layout-aware PDF/Office/OCR parsing, persistent sync, and verifiable data graphs.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
A robust web archive analytics toolkit
- python
- web
- warc
- bigdata