🚀 Easier & Faster YOLO Deployment Toolkit for NVIDIA 🛠️
- tensorrt
- tensorrt-inference
- tensorrt10
- computer-vision
111,085 visible repositories in the index
49–72 of 100 · Most stars
🚀 Easier & Faster YOLO Deployment Toolkit for NVIDIA 🛠️
The inference module for AgiBot X1.
OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
NVIDIA Inference Xfer Library (NIXL)
KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.
LLMs as Copilots for Theorem Proving in Lean
An Industrial Graph Neural Network Framework
solution of exercises of the book "probabilistic robotics"
This is the COVID-19 CovidSim microsimulation model developed by the MRC Centre for Global Infectious Disease Analysis hosted at Imperial College, London.
FeatherCNN is a high performance inference engine for convolutional neural networks.
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
High Performance LLM Inference Operator Library
Local inference engine optimized for NVIDIA Blackwell
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
Generalized Random Forests
Cross-Platform Production-ready C++ inference engine for YOLO models (v5-v12, YOLO26). Unified API for detection, segmentation, pose estimation, OBB, and classification. Built on ONNX Runtime and OpenCV. Optimized for CPU/GPU with quantization support.
Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.
LLM inference in C/C++