Your GeForce runs AI at half speed. This fixes it for 20/30/40 series cards. Up to 2× FP16 / BF16 / TF32 tensor throughput
Repository Index
96,838 visible repositories in the index
1–3 of 3 · Most stars
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
Unlock full speed FP16 tensor core of NVIDIA Turing GPUs