AirLLM 2.8T LLM inference with single 4GB GPU
- chinese-nlp
- finetune
- generative-ai
- instruct-gpt
Repository Index
79,350 visible repositories in the index
1–24 of 44 · Most stars
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
Accessible large language models via k-bit quantization for PyTorch.
- llm
- machine-learning
- pytorch
- qlora
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
The central nervous system and backend orchestrator for Fractal. It handles multi-tenant task scheduling, real-time TFLOPs budgeting, and federated weight aggregation using FedAvg.
A unified inference and post-training framework for accelerated video generation.
- diffusers
- diffusion-models
- video-generation
- distillation
Fast inference engine for Transformer models
- neural-machine-translation
- cpp
- mkl
- quantization
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
- attention
- inference-acceleration
- llm
- quantization
State-of-the-art LLM compression, built for production inference with vLLM
- compression
- quantization
TurboDiffusion: 100–200× Acceleration for Video Diffusion Models
- ai-infra
- diffusion-models
- inference-acceleration
- rcm
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
llama.cpp fork with additional SOTA quants and improved performance
[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.
- pruning
- model-compression
- efficient-deep-learning
- llm
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
- knowledge-distillation
- model-compression
- quantization
- pretrained-models
4 bits quantization of LLaMA using GPTQ
PyTorch native quantization for training and inference
- brrr
- dtypes
- inference
- mx
An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
- deep-learning
- model-compression
- mobile-app
- automl
Images to inference with no labeling (use foundation models to train supervised models).
- computer-vision
- model-distillation
- auto-labeling
- deep-learning
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
- quantization
- deep-learning
- compression
- open-source
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- low-precision
- pruning
- sparsity
- auto-tuning
Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)
- pytorch-tutorial
- pytorch-tutorials
- pytorch
- quantization