MACE is a deep learning inference framework optimized for mobile heterogeneous computing platforms.
- deep-learning
- neural-network
- opencl
- neon
132,241 visible repositories in the index
25–48 of 106 · Most stars
MACE is a deep learning inference framework optimized for mobile heterogeneous computing platforms.
Fast inference engine for Transformer models
MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
Tengine is a lite, high performance, modular inference engine for embedded device
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
校招、秋招、春招、实习好项目!带你从零实现一个高性能的深度学习推理库,支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step
High-performance single-GPU inference for selected model checkpoints and GPUs.
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash (220B) on your own GPUs - from 8 GB of VRAM, NVIDIA and AMD, one dashboard, OpenAI- and Anthropic-compatible API.
Lucebox inference engine for heterogeneous hardware & consumer GPUs
Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details.
Deep Learning Server and CLI for Torch and TensorRT
A local inference engine for Apple silicon, built around the model.
LLM inference in C/C++
ggml speech-to-text inference for 16+ model families
Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.
TinyML AI inference library
🚀 Easier & Faster YOLO Deployment Toolkit for NVIDIA 🛠️
The inference module for AgiBot X1.
OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model