A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
132,241 visible repositories in the index
1–24 of 107 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
- fine-tuning
- llama
- llm
- peft
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- adapter
- diffusion
- llm
- parameter-efficient-learning
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- deepseek-v4
- edge
- glm
- inference
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.
Low-code framework for building custom LLMs, neural networks, and other AI models
Layout-aware document engine for AI and a feature-rich Python toolkit for extracting, rendering and editing PDFs and other documents. Since version 2.0, its ML-based layout analysis turns PDFs and Office files into high-quality Markdown for RAG and agents, running fast on CPU and grounding every result to its source page.
- mupdf
- xps
- pdf-documents
- epub
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
High accuracy RAG for answering questions from scientific documents with citations
Real-time Claude Code usage monitor with predictions and warnings
- ai
- claude
- claude-code
- limits
Infrastructure for continually self‑improving agents
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
- evaluation
- benchmark
- large-language-model
- llm
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
- llmops
- observability
- open-telemetry
- metrics
Efficient Triton Kernels for LLM Training
- llm-training
- triton
- finetuning
- gemma2
Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL
EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
TranslatedDeepAnalyze is the first agentic LLM for autonomous data science. 🎈Your AI data analyst, automatically analyzes large amounts of data and generates professional analysis reports with one click!