A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
65,764 visible repositories in the index
1–24 of 52 · Most stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
- fine-tuning
- llama
- llm
- peft
Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more
- abliteration
- llm
- transformer
- censorship-resistance
SGLang is a high-performance serving framework for large language models and multimodal models.
TranslatedLangchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain
- chatglm
- langchain
- llm
- knowledge-base
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- adapter
- diffusion
- llm
- parameter-efficient-learning
Train transformer language models with reinforcement learning.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
State-of-the-Art Embeddings, Retrieval, and Reranking
Private AI platform for agents, assistants and enterprise search. Built-in Agent Builder, Deep research, Document analysis, Multi-model support, and API connectivity for agents.
- ai
- python
- natural-language-processing
- react
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- deepseek-v4
- edge
- glm
- inference
Nano vLLM
A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
- attention-mechanism
- deep-learning
- gpt
- gpt-2
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
- python
- search
- nlp
- semantic-search
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.
- model
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
- python
- fine-tuning
- gguf
- huggingface
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
- chatgpt
- deep-learning
- instruction-following
- language-model
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
- cuda-kernels
- deepspeed
- fastertransformer
- llm-inference