Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
- knowledge-distillation
- pytorch
- teacher-free
- paper-implementations
142,187 visible repositories in the index
73–79 of 79 · Most stars
Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
A general framework for distilling human-created multimodal resources into reusable, executable skills that AI agents can browse, compose, and run, validated across diverse domains including web, PowerPoint, Excel, Blender, CAD, Unreal Engine 5, and REAPER-based music production.
Official code for "Parser-Free Virtual Try-on via Distilling Appearance Flows", CVPR 2021.
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
a systemone proxy using embeddings that distills a teacher model
[Tech Report] Simplifying and Improving On-Policy Distillation for Few-step Diffusion
Local-first AI trend radar and live-session distillation