A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
132,241 visible repositories in the index
1–24 of 39 · Most stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
- fine-tuning
- llama
- llm
- peft
SGLang is a high-performance serving framework for large language models and multimodal models.
How Python does AI. Agents, realtime voice, image generation, embeddings. Every model, every interface, typed end to end.
- agent-framework
- pydantic
- python
- genai
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- deepseek-v4
- edge
- glm
- inference
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
A Next-Generation Training Engine Built for Ultra-Large MoE Models
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
- machine-learning
- experimentation
- llm-prompting
- llm-test
YouTube Full Text Search - Search all of YouTube from the command line
- chromadb
- full-text-search
- semantic-search
- youtube
Training Sparse Autoencoders on Language Models
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
This repository allows reproduction of Poetiq's record-breaking submission to the ARC-AGI-1 and ARC-AGI-2 benchmarks.
Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.
- prompt-engineering
- llm
- llm-inference
- prompt
AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices.
Quantized inference code for LLaMA models
This Word Does Not Exist
The repository host gives no description.
⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops
- fastapi
- langchain
- langchain-python
- large-language-models
AI agent skill that answers macro questions — housing, gold, BTC, geopolitics — with probability estimates mined from 13 financial data sources (Polymarket, Kalshi, CFTC, SEC & more). For Claude Code / Cursor / Codex / OpenClaw. | 让 AI Agent 从金融数据中挖掘宏观趋势的数字先知
Halo is an open-source framework built by White Circle for training large language and multimodal models
- agent
- async-rl
- distributed-training
- fine-tuning