Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Repository Index
142,273 visible repositories in the index
217–240 of 452 · Most stars
A chat bot for Slack (https://slack.com).
Simple script for downloading Youtube comments without using the Youtube API
- youtube-comments
- youtube
- python
- data-science
批量抓取AV磁链或封面的苦劳力
源码级 Chromium 指纹内核,检测站按普通 Chrome 评分。Source-level Chromium fingerprint kernel that passes bot detection tests. Not JS injection — C++ patches compiled into the binary.
- ai-agents
- anti-detect
- anti-detect-browser
- bot-detection
A tool to scrape LinkedIn without API restrictions for data reconnaissance
DOM Traversing and Scraping using GraphQL
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine
- meta-search-engine
- rust
- search-engine
- fast
A completely revamped and redesigned fork with OnlyFans and Fansly support, reimagined from scratch based on the original onlyfans-scraper.
- datascraping
- downloader
- fansite
- onlyfans
Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr alternative and drop-in replacement for your *arr stack.
an unofficial screen-scraping "API" for Mint.com
Provide scrapers for all major Israeli banks and credit card companies
A self-hosted API that takes a URL and returns a file with browser screenshots.
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
- articles
- articles-data
- crawler
- datasets-preparation
python爬虫
- python
- python3
- urllib
- requests
First ever tool to view "Instagram private posts" anonymously
Automatically archive links to videos, images, and social media content from Google Sheets (and more).
- archive
- docker
- open-source-research
- python
:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
- web-browser
- web-scraper
- web-proxy
- web-filter
Creating Scrapy scrapers via the Django admin interface
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
This is a lightweight, headless browser purpose-built for AI agents (such as Codex or Claude Code) to perform browser automation tasks; it is designed to navigate through various web protection systems, login processes, and reverse-engineering scenarios
- anti-bot
- automation
- firefox
- headless-browser
Scalable Python web scraping scripts for +80 popular domains
Faster requests on Python 3
- python
- python3
- python-library
- http-requests
Crawly, a high-level web crawling & scraping framework for Elixir.