A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
85,578 visible repositories in the index
1–23 of 23 · Most stars
Get your documents ready for gen AI
SGLang is a high-performance serving framework for large language models and multimodal models.
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
Home of StarCoder: fine-tuning & inference!
The RL Bridge for LLM-based Agent Applications. Made Simple & Flexible.
- llm
- llm-reasoning
- machine-learning-systems
- mlsys
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
Let ChatGPT teach your own chatbot in hours with a single GPU!
Community maintained hardware plugin for vLLM on Huawei Ascend
⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡
vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
Home of StarCoder2!
Community maintained hardware plugin for vLLM on Apple Silicon
Running Docling as an API service
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Multimodal RL training framework for diffusion & omni models
- diffusion-models
- flow-matching
- grpo
- multimodal
AutoGroq is a groundbreaking tool that revolutionizes the way users interact with Autogen™ and other AI assistants. By dynamically generating tailored teams of AI agents based on your project requirements, AutoGroq eliminates the need for manual configuration and allows you to tackle any question, problem, or project with ease and efficiency.
- agents
- ai
- artificial-intelligence
- autogen
OWASP Top 10 for Large Language Model Apps (Part of the GenAI Security Project)
Runtime defense against memory poisoning in AI agents (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.
project on fine-tuning qwen to generate text w/o letter 'e'
DeepSeek Harness Desktop 插件，让 AI Agent 用自然语言构建、检查和维护可编辑的 SideFX Houdini 工程，支持程序化建模、HDA、场景诊断与渲染。 | A DeepSeek Harness Desktop plugin that lets AI agents build, inspect, and maintain editable SideFX Houdini projects with natural language, including procedural modeling, HDAs, scene diagnostics, and rendering.
- dsh-plugin
- houdini
- ai-agent
Turn any Java, Kotlin or Python project into an AI-ready context file in one command.