Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
Repository Index
132,241 visible repositories in the index
193–216 of 324 · Most stars
Neural Network Compression Framework for enhanced OpenVINO™ inference
- quantization
- pruning
- sparsity
- quantization-aware-training
Official Inference Code of EAGLE-4
- eagle
- gpu
- llm-inference
- sglang
PaddleOCR inference in PyTorch. Converted from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
A Python package for causal inference in quasi-experimental settings
- causal-inference
- pymc
- quasi-experimental
- quasi-experiments
RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Variational autoencoder implemented in tensorflow and pytorch (including inverse autoregressive flow)
- vae
- machine-learning
- tensorflow
- variational-autoencoder
Deep Object Pose Estimation (DOPE) – ROS inference (CoRL 2018)
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
⚡VoltaML is a lightweight library to convert and run your ML/DL deep learning models in high performance inference runtimes like TensorRT, TorchScript, ONNX and TVM.
Idol: a single image to 3D Gaussians. Inference SDK and demo.
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.
- prompt-engineering
- llm
- llm-inference
- prompt
A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.
Trustworthy AI related projects
- causal-discovery
- causal-inference
- causality
- independence-tests
BlackJAX is a Bayesian Inference library designed for ease of use, speed and modularity.
- probabilistic-programming
- bayesian-inference
- sampling-methods
- hamiltonian-monte-carlo
Recipes to scale inference-time compute of open models
EfficientFormerV2 [ICCV 2023] & EfficientFormer [NeurIPs 2022]
- deep-learning
- detection
- efficient-inference
- efficient-neural-networks
Dataflow compiler for QNN inference on FPGAs
Official inference code for SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis
OpenAlpha_Evolve is an open-source Python framework inspired by the groundbreaking research on autonomous coding agents like DeepMind's AlphaEvolve.
- alphafold
- evolutionary-algorithms
- discovery
Quantized inference code for LLaMA models
Checkpoint-engine is a simple middleware to update model weights in LLM inference engines