🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Repository Index
47,133 repositories indexed
1–24 of 29 · Most stars
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Python scraper based on AI
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
Plumb a PDF for detailed information about each char, rectangle, line, et cetera — and easily extract text and tables.
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications
Module for automatic summarization of text documents and HTML pages.
Malware Configuration And Payload Extraction
Deep neural network to extract intelligent information from invoice documents.
Feature engineering and selection open-source Python library compatible with sklearn.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
:speech_balloon: SpeechPy - A Library for Speech Processing and Recognition: http://speechpy.readthedocs.io/en/latest/
Computational toolbox for large scale Calcium Imaging Analysis, including movie handling, motion correction, source extraction, spike deconvolution and result visualization.
:sound: spafe: Simplified Python Audio Features Extraction
:hospital: Medical Text Mining and Information Extraction with spaCy
A framework for automated extraction of static and dynamic features from Android applications
Model extraction attacks on Machine-Learning-as-a-Service platforms.
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.
Nellie: Automated organelle segmentation, tracking, and hierarchical feature extraction in 2D/3D live-cell microscopy
PDF extraction that audits its own output — and certifies any other extractor's, catching pages they silently dropped. Verify signed manifests offline: free, MIT, no account. 0.903 on opendataloader-bench, #2 of 8 engines. 7-tool MCP server.
AI-powered platform for OSINT intelligence analysis. Features archive discovery with hypothesis-driven investigation, GLiNER entity extraction, Mapbox geospatial visualization, network analysis, and document processing. Built with FastAPI, Next.js, Weaviate, and DSPy.