Tensor library for machine learning
Repository Index
90,854 visible repositories in the index
1–24 of 33 · Most stars
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
Accessible large language models via k-bit quantization for PyTorch.
- llm
- machine-learning
- pytorch
- qlora
2bit quants, kernels and runtime for consumer devices.
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
The central nervous system and backend orchestrator for Fractal. It handles multi-tenant task scheduling, real-time TFLOPs budgeting, and federated weight aggregation using FedAvg.
A unified inference and post-training framework for accelerated video generation.
- diffusers
- diffusion-models
- video-generation
- distillation
Fast inference engine for Transformer models
- neural-machine-translation
- cpp
- mkl
- quantization
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
- attention
- inference-acceleration
- llm
- quantization
TurboDiffusion: 100–200× Acceleration for Video Diffusion Models
- ai-infra
- diffusion-models
- inference-acceleration
- rcm
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
Images to inference with no labeling (use foundation models to train supervised models).
- computer-vision
- model-distillation
- auto-labeling
- deep-learning
An Open-source Toolkit for LLM Development
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
- quantization
- deep-learning
- compression
- open-source
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
OneDiff: An out-of-the-box acceleration library for diffusion models.
- comfyui
- diffusers
- pytorch
- sdxl
TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration
Pytorch implementation of various Knowledge Distillation (KD) methods.
- kd-methods
- knowledge-transfer
- knowledge-distillation
- kd
PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
- int4
- quantization
- rounding
- transformers
[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
This repository provides code for machine learning algorithms for edge devices developed at Microsoft Research India.
- iot-device
- sensor
- edge-machine-learning
- resource-constrained-ml