🧑🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
A treasure chest for visual classification and recognition powered by PaddlePaddle
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
A unified inference and post-training framework for accelerated video generation.
Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.
TurboDiffusion: 100–200× Acceleration for Video Diffusion Models
DAMO-YOLO: a fast and accurate object detection method with some new techs, including NAS backbones, efficient RepGFPN, ZeroHead, AlignedOTA, and distillation enhancement.
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
- knowledge-distillation
- model-compression
- quantization
- pretrained-models
"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)
- controlnet
- knowledge-distillation
- pose-estimation
- stable-diffusion-webui
Images to inference with no labeling (use foundation models to train supervised models).
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- low-precision
- pruning
- sparsity
- auto-tuning
[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods
The official repo for [NeurIPS'22] "ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation" and [TPAMI'23] "ViTPose++: Vision Transformer for Generic Body Pose Estimation"
EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR, BI-R, ER, A-GEM, iCaRL, Generative Classifier) in three different scenarios.
- deep-learning
- artificial-neural-networks
- continual-learning
- lifelong-learning
Pytorch implementation of various Knowledge Distillation (KD) methods.
- kd-methods
- knowledge-transfer
- knowledge-distillation
- kd
All-in-one training for vision models (YOLO, ViTs, RT-DETR, DINOv3): pretraining, fine-tuning, distillation.
A PyTorch-based knowledge distillation toolkit for natural language processing
OpenMMLab Model Compression Toolbox and Benchmark.
- nas
- pruning
- knowledge-distillation
- spos
A coding-free framework built on PyTorch for reproducible deep learning studies. PyTorch Ecosystem. 🏆28 knowledge distillation methods presented at TPAMI, CVPR, ICLR, ECCV, NeurIPS, ICCV, AAAI, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
PaddleSlim is an open-source library for deep model compression and architecture search.
- pruning
- quantization
- nas
- bert
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (NeurIPS 2023 Spotlight)
- diffusion-model
- dreamfusion
- nerf
- text-to-3d
yolov3 yolov4 channel and layer pruning, Knowledge Distillation 层剪枝,通道剪枝,知识蒸馏