🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Convert PDF to markdown + JSON quickly with high accuracy
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
Python scraper based on AI
Give Claude the ability to watch any video. /watch downloads, extracts frames, transcribes, hands it all to Claude.
Replace 'hub' with 'ingest' in any GitHub URL to get a prompt-friendly extract of a codebase
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
OCR model that handles complex tables, forms, handwriting with full layout.
- ai
- ocr
- document-ai
- document-parsing
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
- mupdf
- xps
- pdf-documents
- epub
Plumb a PDF for detailed information about each char, rectangle, line, et cetera — and easily extract text and tables.
- pdf
- pdf-parsing
- table-extraction
A Comprehensive Toolkit for High-Quality PDF Content Extraction
TranslatedA GUI tool for extracting hard-coded subtitle (hardsub) from videos and generating srt files. Extracts hard subtitles from videos and generates srt files. No need to apply for a third-party API; text recognition is done locally. A deep-learning-based video subtitle extraction framework, including subtitle region detection and subtitle content extraction.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.
A PyTorch implementation of EfficientNet
- efficientnet-pytorch
- imagenet
- feature-extraction
- pretrained-models
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
An open source library for deep learning end-to-end dialog systems and chatbots.
Unified real-time search engine skill for AI agents. Supports general web search, vertical domain search, parallel batch search, and full-page content extraction.
PDF craft can convert PDF files into various other formats. This project will focus on processing PDF files of scanned books.