A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models
Repository Index
142,187 visible repositories in the index
73–96 of 123 · Most stars
A minimalist SOTA LaTeX OCR model with only 20M parameters, running in browser. Full training pipeline available for self-reproduction. | 超轻量SOTA LaTeX公式识别模型,仅20M参数量,可在浏览器中运行。训练全流程代码开源,以便自学复现。
- computer-vision
- deep-learning
- distillation-model
- hydra
Official implementation for "Towards a Unified Self-Distillation Framework for Large Language Models" (https://arxiv.org/abs/2605.06597).
Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline
Awesome List for On-Policy Distillation
- awesome-list
- distillation
- large-language-model
- on-policy-distillation
EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
- bert
- nlp-applications
- knowledge-distillation
- transfer-learning
mobilev2-yolov5s剪枝、蒸馏,支持ncnn,tensorRT部署。ultra-light but better performence!
- mobile-yolov5s
- distillation
- pruning
- ncnn
An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning
- pytorch
- continual-learning
- neurips2020
- knowledge-distillation
sanoTTS (सानो = 'small' in Nepali): 294k–2.27M-parameter neural TTS that runs on a $3 chip, in the browser, or in Python with low-latency streaming.
Kandinsky 5.0: A family of diffusion models for Video & Image generation
- diffusion
- distillation
- kandinsky
- text-to-video
rCM & Causal-rCM: Leading and Unified Algorithms/Infrastructures for Bidirectional/Autoregressive Video Diffusion Distillation at Scale
- consistency-model
- diffusion
- distillation
- jacobian-vector-product
Open-source code for paper "Dataset Distillation"
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
高质量中文预训练模型集合:最先进大模型、最快小模型、相似度专门模型
- pretrained-models
- chinese
- bert
- roberta
A source-available Codex skill for distilling photographs into sparse editorial abstractions.
- codex
- codex-skill
- image-generation
- visual-analysis
【小红书抖音3万人看过的——博主蒸馏器】你研究了 N 个爆款博主,还是说不清任何一个的操作系统。输入博主的链接,直接蒸馏 TA 的认知和内容打法,装进你的 AI。Distill any blogger's mind into your AI.
An open-source platform for long-duration multi-agent workflows. This project automates long-running, multi-agent AI work by distilling each agent’s output into context for the team of agents. You can let the distilled context to drive each agent.I hope this project can save people's time and be cost effcient- since Deepseek models are cheaper.
Get FAR MORE done with FAR FEWER tokens 🔥
- claude-code
- codex
- llm
- tokens
knowledge distillation papers
- knowledge-distillation
- model-compression
- paper
- dark-knowledge
A curated catalog of human distillliation agent skills
- agent-skills
- awesome-list
- claude-code
- claude-code-skills
SparkDiffusion: Accelerating DiT video generation by 265× via joint Sparsity, Distillation, Quantization, etc.
- diffusion-models
- distillation
- high-sparsity
- inference-acceleration
The official code for the paper 'Structured Knowledge Distillation for Semantic Segmentation'. (CVPR 2019 ORAL) and extension to other tasks.
The repository host gives no description.
[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices
- image-inpainting
- knowledge-distillation
- object-removal
- mi-gan