Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Repository Index
78,351 visible repositories in the index
1–14 of 14 · Most stars
PyTorch native post-training library
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
The central nervous system and backend orchestrator for Fractal. It handles multi-tenant task scheduling, real-time TFLOPs budgeting, and federated weight aggregation using FedAvg.
State-of-the-art LLM compression, built for production inference with vLLM
- compression
- quantization
llama.cpp fork with additional SOTA quants and improved performance
Images to inference with no labeling (use foundation models to train supervised models).
- computer-vision
- model-distillation
- auto-labeling
- deep-learning
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- low-precision
- pruning
- sparsity
- auto-tuning
Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)
- pytorch-tutorial
- pytorch-tutorials
- pytorch
- quantization
[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
- pytorch
- knowledge-distillation
- deep-neural-networks
- cifar10
OneDiff: An out-of-the-box acceleration library for diffusion models.
- comfyui
- diffusers
- pytorch
- sdxl
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~41 in fast mode, 512K context. Plus GLM-5.3-Flash, Qwen3.8 and LFM3-Omni. MoE experts tiered across VRAM, pinned RAM and NVMe, with Xolotl Direct NVMe->VRAM. OpenAI + Anthropic API.
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
- int4
- large-language-models
- pytorch
- quantization