Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.
- huggingface
- inference
- llm
- mechanistic-interpretability
132,241 visible repositories in the index
433–456 of 748 · Most stars
Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.
High Performance LLM Inference Operator Library
Deep Object Pose Estimation (DOPE) – ROS inference (CoRL 2018)
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
⚡VoltaML is a lightweight library to convert and run your ML/DL deep learning models in high performance inference runtimes like TensorRT, TorchScript, ONNX and TVM.
Idol: a single image to 3D Gaussians. Inference SDK and demo.
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.
A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.
Trustworthy AI related projects
BlackJAX is a Bayesian Inference library designed for ease of use, speed and modularity.
Recipes to scale inference-time compute of open models
Generalized Random Forests
EfficientFormerV2 [ICCV 2023] & EfficientFormer [NeurIPs 2022]
Lightweight, useful implementation of conformal prediction on real data.
Cross-Platform Production-ready C++ inference engine for YOLO models (v5-v12, YOLO26). Unified API for detection, segmentation, pose estimation, OBB, and classification. Built on ONNX Runtime and OpenCV. Optimized for CPU/GPU with quantization support.
Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.
Dataflow compiler for QNN inference on FPGAs
Run Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime
RStan, the R interface to Stan
AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sandbox project.
Official inference code for SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis
A research prototype for zero-knowledge proofs of AI inference. Not maintained.