Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook — no data leaves your device.Mano-P 是一个开源 GUI-VLA 项目,支持在 Mac mini/MacBook 上或通过算力棒本地运行推理,实现纯视觉驱动的跨平台 GUI 自动化操作。数据完全本地处理,支持复杂多步骤任务规划与执行。
- computer-use-agents
- desktop-automation
- edge-computing
- gui-automation
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.
- apple-silicon
- local-ai
- mlx
- mtp
Use Hugging Face with JavaScript
Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
A local inference engine for Apple silicon, built around the model.
LLM inference in C/C++
Manages Unified Access to Generative AI Services built on Envoy Gateway
LLM Inference Engine for Metal, CUDA and Vulkan.
Communicate with an LLM provider using a single interface
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
ggml speech-to-text inference for 16+ model families
Inference Llama 2 in one file of pure 🔥
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
AICI: Prompts as (Wasm) Programs
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
14-stage Fusion Pipeline for LLM token compression — reversible compression, AST-aware code analysis, intelligent content routing. Zero LLM inference cost. MIT licensed.
- ai-agent-tools
- context-compression
- context-pruning
- context-window-optimization
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Large-scale LLM inference engine
Tile-Based Runtime for Ultra-Low-Latency LLM Inference
The non profit fostering AI in India through credits, grants, resources
Unified, efficient, and easy-to-use inference infrastructure for edge AI || 面向端侧 AI 的统一、高效、易用的推理基础设施
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs