Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
Repository Index
109,091 visible repositories in the index
1–24 of 64 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Convert PDF to markdown + JSON quickly with high accuracy
- deep-learning
- document-conversion
- document-parsing
- docx
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
PDF Parser for AI-ready data. Automate PDF accessibility. Open-source.
A fast, helpful, and open-source document parser
- document-ocr
- document-processing
- ocr
- ocr-recognition
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
OCR model that handles complex tables, forms, handwriting with full layout.
OCR & Document Extraction using vision models
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
- mupdf
- xps
- pdf-documents
- epub
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
- text-extraction
- document-intelligence
- metadata-extraction
- pdf-extraction
Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM
- machinelearning
- huggingface-transformers
- computer-vision
- llm
Extrapolating knowledge graphs from unstructured text using GPT-3 🕵️♂️
- gpt-3
- knowledge-graph
Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
The Apache Tika toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF).
- java
- tika
- metadata
- extraction
Knowhere extracts, parses, and outputs structured chunks ready for AI Agents and RAG.
- ai-agents
- claude-code
- cursor
- langchain
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
- machine-learning
- convolutional-networks
- embedding-vectors
- embeddings
Document (PDF, Word, PPTX ...) extraction and parse API using state of the art modern OCRs + Ollama supported models. Anonymize documents. Remove PII. Convert any document or picture to structured JSON or Markdown
QuantMind is an open source agent-native knowledge extraction and retrieval framework for quantitative finance.
- quantitative-finance
- quantitative-research
- data
- knowledge
NeMo Retriever Library is a scalable, performance-oriented document content and metadata extraction microservice. NeMo Retriever Library uses specialized NVIDIA NIM microservices to find, contextualize, and extract text, tables, charts and images that you can use in downstream generative applications.
A curated, battle-tested AI-coding toolkit: Claude Code plugins, subagent orchestration, quality gates, and ready-to-copy prompts, extracted from real production use.