🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications
a library for audio and music analysis
Module for automatic summarization of text documents and HTML pages.
A library for audio and music analysis, feature extraction.
Feature engineering and selection open-source Python library compatible with sklearn.
The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5× faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.
:speech_balloon: SpeechPy - A Library for Speech Processing and Recognition: http://speechpy.readthedocs.io/en/latest/
:sound: spafe: Simplified Python Audio Features Extraction
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.
A robust web archive analytics toolkit
PDF extraction that audits its own output — and certifies any other extractor's, catching pages they silently dropped. Verify signed manifests offline: free, MIT, no account. 0.903 on opendataloader-bench, #2 of 8 engines. 7-tool MCP server.
AI-powered platform for OSINT intelligence analysis. Features archive discovery with hypothesis-driven investigation, GLiNER entity extraction, Mapbox geospatial visualization, network analysis, and document processing. Built with FastAPI, Next.js, Weaviate, and DSPy.
AI-Driven Full-Text Screening and Data Extraction for Systematic Reviews and Evidence Synthesis
A lightweight Python package for teaching computational methods through art history. Extract and analyze artist-specific works from WikiArt with built-in visualizations. Designed for digital humanities and computational design courses. Features genre/style analysis, matplotlib integration, and pedagogical clarity over algorithmic complexity.