Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Repository Index
110,088 visible repositories in the index
25–48 of 64 · Most stars
The official CLI for Agentic Document Extraction (ADE) by LandingAI — parse documents and extract schema-shaped data from your terminal
⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
- nlp
- text-classification
- text-matching
- information-extraction
Extract structured data from PDF invoices
- python
- data-mining
Extract structured data and insights from documents using LLMs and decision models
- ai
- data-extraction
- document-intelligence
- generative-ai
An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)
- document
- document-analysis
- extraction
- llms
Generate UI screens from text prompts and extract their HTML and screenshots programmatically.
Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.
- extraction
- tika
- unstructured
An easy way to extract information from documents
Synthetic data curation for post-training and structured data extraction
node.js module for extracting text from html, pdf, doc, docx, xls, xlsx, csv, pptx, png, jpg, gif, rtf and more!
- extract-text
- extraction
- nodejs
LLM(😽)
- information-extraction
- llm
- natural-language
- natural-language-processing
Tika-Python is a Python binding to the Apache Tika™ REST services allowing Tika to be called natively in the Python community.
- tika-server
- python
- tika-python
- tika-server-jar
ExtractThinker is a Document Intelligence library for LLMs, offering ORM-style interaction for flexible and powerful document workflows.
Extract and convert data from any document, images, pdfs, word doc, ppt or URL into multiple formats (Markdown, JSON, CSV, HTML) with intelligent structured data extraction and advanced OCR.
Easily deployable and scalable backend server that efficiently converts various document formats (pdf, docx, pptx, html, images, etc) into Markdown. With support for both CPU and GPU processing, it is Ideal for large-scale workflows, it offers text/table extraction, OCR, and batch processing with sync/async endpoints.
- api
- fastapi
- markdown-parser
- pdf-chatbot
Open-source platform for extracting structured data from documents using AI.
Colivara is a suite of services that allows you to store, search, and retrieve documents based on their visual embedding. ColiVara has state of the art retrieval performance on both text and visual documents. using vision models instead of chunking and text-processing for documents. No OCR, no text extraction, no broken tables, or missing images.
An elegent pytorch implement of transformers
Extract structured data from documents quickly and accurately.
📄 PDF extraction and rendering across all JavaScript runtimes
- pdfjs
- serverless
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
A MemAgent framework that can be extrapolated to 3.5M, along with a training framework for RL training of any agent workflow.
Local models support for Microsoft's graphrag using ollama (llama3, mistral, gemma2 phi3)- LLM & Embedding extraction