Production ready toolkit to run AI locally
Repository Index
132,241 visible repositories in the index
1–23 of 23 · Most stars
TranslatedPaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge
On-device Speech AI for Apple Silicon
- inference
- ios
- speech-recognition
- swift
🔥🔥🔥AidLearning is a powerful AIOT development platform, AidLearning builds a linux env supporting GUI, deep learning and visual IDE on Android...Now Aid supports CPU+GPU+NPU for inference with high performance acceleration...Linux on Android or HarmonyOS
Run Stable Diffusion on Android Devices with Snapdragon NPU acceleration. Also supports CPU/GPU inference.
- stable-diffusion
- stable-diffusion-android
- img2img
- inpainting
Frontier CoreML audio models in your apps — text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source.
Unified, efficient, and easy-to-use inference infrastructure for edge AI || 面向端侧 AI 的统一、高效、易用的推理基础设施
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
- computer-vision
- executorch
- llm-inference
- object-detection
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference.
- gemma
- ios
- litert
- local-agent
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
- android
- cpp
- edge-ai
- gguf
Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.
- android
- android-17
- edge-ai
- jetpack-compose
⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
- local-ai
- ai-sdk
- decentralized-ai
- llama
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability
Real-time portrait segmentation for mobile devices
- portrait-segmentation
- keras
- tensorflow
- android
Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source
- kotlin-android
- litert
- litert-lm
- llm
🏹 Parse JSON with style
- json
- json-parsing
- ios
- swift
MobiInfer runs 35B-A3B-class MoE models fully on-device on the Qualcomm Hexagon NPU, streaming experts from flash with lookahead scheduling (LES) and DSpark speculation.
PocketStrike is a privacy-first mobile AI workstation. It combines zero-cloud native on-device GGUF inference (ARM NEON accelerated) │ with frontier cloud LLMs, multi-step ReAct agent tools, Model Context Protocol (MCP) servers, self-evolving cross-session memory, and FLUX neural image │ generation.
Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app.
🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift
OtosakuStreamingASR-iOS is a real-time speech recognition engine for iOS, built with Swift and Core ML. It uses a fast and lightweight streaming Conformer model optimized for on-device inference. Designed for developers who need efficient audio transcription on mobile.
- asr
- conformer
- coreml
- ios