Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–13 of 13 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
- text-extraction
- document-intelligence
- metadata-extraction
- pdf-extraction
Free database schema discovery and comprehension tool
- schemacrawler
- jdbc
- database-diagrams
- schema
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5× faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.
- document-processing
- pdf-editor
- pdf-generation
Rust web scraper & crawler for AI agents: URLs, PDFs and Office files → clean Markdown/JSON. Firecrawl- and Jina Reader-compatible API, MCP server, Python/Node SDKs. Self-hosted single binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
Tiny ISO-compliant C++ EXIF and XMP parsing library for JPEG.
- exif
- exif-metadata
- exif-data-extraction
- exif-reader
AI-Driven Full-Text Screening and Data Extraction for Systematic Reviews and Evidence Synthesis
- academic-research
- ai-assistant
- ai-tool
- data-extraction