Sign in

Repository Index

89,572 visible repositories in the index

Tags: All Tags
Tags

A repository shows when it has any selected Tag.

Filters (1)

Filters

Clear all filters

1–7 of 7 · Most stars

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

  • attention
  • inference-acceleration
  • llm
  • quantization

This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.

  • gpu-acceleration
  • elementwise
  • reduce
  • sgemm