A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–24 of 31 · Most stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more
- abliteration
- llm
- transformer
- censorship-resistance
SGLang is a high-performance serving framework for large language models and multimodal models.
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- adapter
- diffusion
- llm
- parameter-efficient-learning
Train transformer language models with reinforcement learning.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
State-of-the-Art Embeddings, Retrieval, and Reranking
Private AI platform for agents, assistants and enterprise search. Built-in Agent Builder, Deep research, Document analysis, Multi-model support, and API connectivity for agents.
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- deepseek-v4
- edge
- glm
- inference
Nano vLLM
A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
- attention-mechanism
- deep-learning
- gpt
- gpt-2
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
- python
- search
- nlp
- semantic-search
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
- python
- fine-tuning
- gguf
- huggingface
An open, programmable decision layer for models and compute.
- mixture-of-models
- semantic-router
- vllm
- ai-gateway
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
- llava
- llm
- mlx
- vision-transformer
Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)
- calibration
- decision-model
- jev
- jev-model
ProtTrans is providing state of the art pretrained language models for proteins. ProtTrans was trained on thousands of GPUs from Summit and hundreds of Google TPUs using Transformers Models.
This Word Does Not Exist