A no-nonsense CAPTCHA system with seamless UX | Backend component
Repository Index
132,241 visible repositories in the index
1–12 of 12 · Most stars
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
A high-level API for programmatically interacting with web pages through WebDriver.
- rust
- webdriver
- automation
- library
Fast proxy scraper and checker written in Rust. Collects HTTP, SOCKS4 and SOCKS5 proxies from any number of sources, verifies each one really works, and writes JSON and plain text with response time, exit IP, ASN and geolocation. Single binary for Windows, Linux, macOS, Android, FreeBSD, NetBSD, illumos and Solaris, with an interactive TUI.
- proxyscraper
- proxyscrape
- proxy-checker
- proxy
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine
- meta-search-engine
- rust
- search-engine
- fast
A Rust library to extract useful data from HTML documents, suitable for web scraping.
operative framework is a rust investigation OSINT framework, you can interact with multiple targets, execute multiple modules, create links with target, export rapport to PDF file, add note to target or results, interact with RESTFul API, write your own modules.
- fingerprint
- framework
- gathering
Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.
Local-first web & document ingestion engine in Rust — HTTP/browser capture, layout-aware PDF/Office/OCR parsing, persistent sync, verifiable data graphs, and evidence-grounded AI context.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
🩵 Self-hosted & Fully-automated ActivityPub Bridge for Static Sites.
- activitypub
- federation
- fediverse
- rust
A robust web archive analytics toolkit
- python
- web
- warc
- bigdata