Accurate, large-scale, and extensible simulator for LLM inference Systems
Repository Index
132,241 visible repositories in the index
265–288 of 324 · Most stars
[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models
- acceleration
- diffusion-models
- generative-ai
- generative-model
yolort is a runtime stack for yolov5 on specialized accelerators such as tensorrt, libtorch, onnxruntime, tvm and ncnn.
- libtorch
- yolov5
- inference
- torchscript
[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization
- efficient-inference
- large-language-models
- llm
- model-compression
[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
- deep-learning
- neural-architecture-search
- pytorch
- tinyml
Serverless LLM Serving for Everyone.
- large-language-models
- model-as-a-service
- model-serving
- serverless-inference
On-device Speech-to-Intent engine powered by deep learning
- natural-language-understanding
- voice-recognition
- nlu
- spoken-language-understanding
v objective diffusion inference code for PyTorch.
LLMFlows - Simple, Explicit and Transparent LLM Apps
- ai
- llm
- llm-inference
- llmops
Bayesian Python: Bayesian inference tools for Python
USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
- attention-is-all-you-need
- llm-inference
- llm-training
- pytorch
TensorFlow models accelerated with NVIDIA TensorRT
- tensorflow
- tensorrt
- nvidia
- jetson
Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.
Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference.
- c99
- catboost
- compiler
- decision-trees
Official inference code for BluePixel, a native 4K image editing model with 4-step distilled inference, built on FLUX.2 Klein and DCAE.
- diffusion-models
- flux
- image-editing
- image-to-image
A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)
Real-time 3D multi-person pose estimation demo in PyTorch. OpenVINO backend can be used for fast inference on CPU.
- human-pose-estimation
- real-time
- 3d-human-pose
- multi-person-pose-estimation
Python Causal Impact Implementation Based on Google's R Package. Built using TensorFlow Probability.
- causalimpact
- tensorflow-probability
- python
- causal-inference
Gaussian processes in JAX and Equinox.
- gaussian-processes
- jax
- probabilistic-programming
- bayesian-inference
[NeurIPS 2022] Denoising Diffusion Restoration Models -- Official Code Repository
- diffusion
- deblurring
- diffusion-models
- inpainting
DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference
Sybil-resistant AI agent network with hardware-attested identity. Proof-of-Antiquity blockchain: physical machines across 15+ CPU architectures prove they are real silicon, not VM farms. Agent economy, micropayments, Solana bridge (wRTC). $0 VC.
- blockchain
- cryptocurrency
- mining
- vintage-hardware
LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis
- llama
- llm
- text-to-speech
- xcodec2
PyTensor allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays.
- ai
- bayesian-inference
- computational-science
- deep-learning