MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
Repository Index
132,241 visible repositories in the index
145–168 of 748 · Most stars
RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.
- ai
- ai-ocr
- gnn
- graph
A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation
- chinese
- english
- multilingual
- realtime
Next generation of automated data exploratory analysis and visualization platform.
- datamining
- visualization
- automated-visualization
- data-exploration
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
- deep-learning
- mnn
- ncnn
- inference
A fast inference library for running LLMs locally on modern consumer-class GPUs
Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
Coz: Causal Profiling
- profiler
- optimization
- causal-inference
- performance-analysis
Tengine is a lite, high performance, modular inference engine for embedded device
TypeDB: Built for systems, not records
- database
- knowledge-base
- knowledge-representation
- reasoning
Pre-trained Deep Learning models and demos (high quality and extremely fast)
- models
- caffemodel
- demo
- tensorflow-models
Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.
- convolutional-neural-networks
- efficient-inference
- imagenet
- model-compression
A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
- ai
- amd-gpu
- apple-silicon
- asr
Optimizing inference proxy for LLMs
- agent
- agentic-ai
- agentic-workflow
- agents
Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
- gpu-acceleration
- large-language-models
- llm
- llm-inference
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
- apple-silicon
- local-llm
- macos
- mlx
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
CSGHub is a brand-new open-source platform for managing LLMs, developed by the OpenCSG team. It offers both open-source and on-premise/SaaS solutions, with features comparable to Hugging Face. Gain full control over the lifecycle of LLMs, datasets, and agents, with Python SDK compatibility with Hugging Face. Join us! ⭐️
- ai
- huggingface
- llm
- management-system
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
- federated-learning
- deep-learning
- distributed-training
- edge-ai
Run Stable Diffusion on Android Devices with Snapdragon NPU acceleration. Also supports CPU/GPU inference.
- stable-diffusion
- stable-diffusion-android
- img2img
- inpainting
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
- attention
- inference-acceleration
- llm
- quantization