FlashInfer: Kernel Library for LLM Serving
Repository Index
106,102 visible repositories in the index
1–8 of 8 · Most stars
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
- attention
- inference-acceleration
- llm
- quantization
FlashKDA: high-performance Kimi Delta Attention kernels
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
- ai-infra
- attention
- inference-acceleration
- llm
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
- huggingface-transformers
- language-vision
- multimodal
- pytorch
Flash Attention in ~100 lines of CUDA (forward pass only)
Fork of SageAttention for Windows wheels and easy installation
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
- attention
- blackwell
- cuda-kernels
- flash-attention