A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
132,241 visible repositories in the index
1–24 of 163 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
- fine-tuning
- llama
- llm
- peft
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- adapter
- diffusion
- llm
- parameter-efficient-learning
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- deepseek-v4
- edge
- glm
- inference
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
The Cloud Native AI Platform
- ml
- kubernetes
- minikube
- tensorflow
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.
Low-code framework for building custom LLMs, neural networks, and other AI models
Layout-aware document engine for AI and a feature-rich Python toolkit for extracting, rendering and editing PDFs and other documents. Since version 2.0, its ML-based layout analysis turns PDFs and Office files into high-quality Markdown for RAG and agents, running fast on CPU and grounding every result to its source page.
- mupdf
- xps
- pdf-documents
- epub
QLoRA: Efficient Finetuning of Quantized LLMs
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
High accuracy RAG for answering questions from scientific documents with citations
Real-time Claude Code usage monitor with predictions and warnings
- ai
- claude
- claude-code
- limits
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
Infrastructure for continually self‑improving agents
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
- evaluation
- benchmark
- large-language-model
- llm
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
- llmops
- observability
- open-telemetry
- metrics
Efficient Triton Kernels for LLM Training
- llm-training
- triton
- finetuning
- gemma2