LLM training in simple, raw C/CUDA
Repository Index
109,091 visible repositories in the index
1–15 of 15 · Most stars
Instant neural graphics primitives: lightning fast NeRF and more
FlashInfer: Kernel Library for LLM Serving
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
- attention
- inference-acceleration
- llm
- quantization
[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.
- pytorch
- acceleration
This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.
- gpu-acceleration
- elementwise
- reduce
- sgemm
Fast CUDA matrix multiplication from scratch
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
- ai-infra
- attention
- inference-acceleration
- llm
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
- huggingface-transformers
- language-vision
- multimodal
- pytorch
Training materials associated with NVIDIA's CUDA Training Series (www.olcf.ornl.gov/cuda-training-series/)
RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
- anns
- building-blocks
- clustering
- cuda
[CVPR 2023 Highlight] Neural Kernel Surface Reconstruction
- 3d-reconstruction
- computer-vision
- graphics
- neural-kernel
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
- attention
- blackwell
- cuda-kernels
- flash-attention
Examples demonstrating available options to program multiple GPUs in a single node or a cluster
cuVS - a library for vector search and clustering on the GPU
- anns
- clustering
- cuda
- distance