Repository Index

A high-throughput and memory-efficient inference and serving engine for LLMs

• gpt
• llm
• pytorch
• model-serving

Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.

• ocr
• chineseocr
• pdf2markdown
• pp-ocr

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

• fine-tuning
• llama
• llms
• gemma

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

• fine-tuning
• llama
• llm
• peft

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

• llm
• nlp
• python
• gemini-ai

A modular graph-based Retrieval-Augmented Generation (RAG) system

• graphrag
• rag
• llm
• llms

The #1 AI Harness for Building Resumes, PDFs, Cover Letters & more, locally with 100+ LLMs support.

• text-similarity
• natural-language-processing
• word-embeddings
• python

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

• machine-learning
• ai
• ml
• mlflow

An open-source RAG-based tool for chatting with your documents.

• chatbot
• llms
• open-source
• rag

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

• llm
• pandas
• ai
• data-analysis

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

TranslatedChinese LLaMA & Alpaca LLMs + local CPU/GPU training and deployment

• llm
• plm
• pre-trained-language-models
• alpaca

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

• llm
• lora
• sft
• multimodal

Structured Outputs

• generative-ai
• llms
• prompt-engineering
• symbolic-ai

GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.

• chatbot
• chatgpt-api
• chatglm
• claude

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

• blackwell
• cuda
• moe
• pytorch

structured outputs for llms

• openai
• python
• pydantic-v2
• openai-functions

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

• ai
• artificial-intelligence
• deep-learning
• large-language-models

TranslatedChatGLM3 series: Open Bilingual Chat LLMs

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

• api
• llm
• api-documentation
• chatgpt

Access large language models from the command-line

• llms
• openai
• ai

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

• llm
• llmops
• model-inference
• fine-tuning

Retrieval and Retrieval-augmented LLMs

• embeddings
• information-retrieval
• llm
• sentence-embeddings