[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
- ai-infra
- attention
- inference-acceleration
- llm
111,821 visible repositories in the index
553–576 of 872 · Most stars
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
LLM KV cache compression made easy
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
Pytorch-Named-Entity-Recognition-with-BERT
Official inference code for SoulX-LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memory
Anatomy of Matplotlib -- tutorial developed for the SciPy conference
android framework层源码解析,不定时更新哈
Added vLLM support to IndexTTS for faster inference.
Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.
A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.
All-in-one RVC song cover toolkit for Google Colab: pull audio from YouTube, separate vocals, train a model, and run inference.
The simplest way to serve AI/ML models in production
ezkl is an engine for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML). Use it from Python, Javascript, or the command line.
https://wavespeed.ai/ [WIP] The all in one inference optimization solution for ComfyUI, universal, flexible, and fast.
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
This is the COVID-19 CovidSim microsimulation model developed by the MRC Centre for Global Infectious Disease Analysis hosted at Imperial College, London.
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )
FeatherCNN is a high performance inference engine for convolutional neural networks.
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
Deep Learning-based Image Fusion: A Survey
Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
Neural Network Compression Framework for enhanced OpenVINO™ inference
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.