Added vLLM support to IndexTTS for faster inference.
Repository Index
132,241 visible repositories in the index
409–432 of 748 · Most stars
A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.
- learning-resources
- llm
- llm-inference
- mlops
Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.
- causal-inference
- graph
- causality
- causal-models
All-in-one RVC song cover toolkit for Google Colab: pull audio from YouTube, separate vocals, train a model, and run inference.
- ai
- rvc
- svc
- song-cover
ezkl is an engine for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML). Use it from Python, Javascript, or the command line.
- ai
- cryptography
- zero-knowledge
- zkml
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )
The simplest way to serve AI/ML models in production
- machine-learning
- artificial-intelligence
- easy-to-use
- inference-api
https://wavespeed.ai/ [WIP] The all in one inference optimization solution for ComfyUI, universal, flexible, and fast.
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
FeatherCNN is a high performance inference engine for convolutional neural networks.
- convolutional-neural-networks
- inference-engine
- caffe
- android
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
- asr
- ctranslate2
- huggingface
- whisper
Neural Network Compression Framework for enhanced OpenVINO™ inference
- quantization
- pruning
- sparsity
- quantization-aware-training
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
- anthropic-api
- cuda
- gemma
- gguf
Official Inference Code of EAGLE-4
- eagle
- gpu
- llm-inference
- sglang
PaddleOCR inference in PyTorch. Converted from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
A Python package for causal inference in quasi-experimental settings
- causal-inference
- pymc
- quasi-experimental
- quasi-experiments
A curated list of causal inference libraries, resources, and applications.
- awesome
- causal-inference
RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
Agent Mesh
- agentic-ai
- p2p-network
- network-security
- networking-ai
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM
- ggml
- kv-cache
- llama-cpp
- llm-inference
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Variational autoencoder implemented in tensorflow and pytorch (including inverse autoregressive flow)
- vae
- machine-learning
- tensorflow
- variational-autoencoder