A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–12 of 12 · Most stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- adapter
- diffusion
- llm
- parameter-efficient-learning
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.
QLoRA: Efficient Finetuning of Quantized LLMs
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Cleora AI is a general-purpose open-source model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data. Created by Synerise.com team.
- ai
- graphs
- synerise
- embeddings
A Unified Framework for Efficient RSI Research
- agent
- recursive-self-improvement
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.