Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
- agent
- anthropic-api
- apple-silicon
- claude-code
132,241 visible repositories in the index
241–264 of 748 · Most stars
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
High-efficiency floating-point neural network inference operators for mobile, server, and Web
Turn any computer or edge device into a command center for your computer vision projects.
A local inference engine for Apple silicon, built around the model.
A framework for integrated Artificial Intelligence & Artificial General Intelligence (AGI)
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
Self-hosted realtime soundscape analyser for birds, bats and other wildlife. Multi-model local AI inference, runs 24/7 on a Raspberry Pi.
LLM inference in C/C++
SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)
Executable file for VITS inference
Manages Unified Access to Generative AI Services built on Envoy Gateway
Mastering Applied AI, One Concept at a Time
LLM Inference Engine for Metal, CUDA and Vulkan.
An agentic company research tool powered by LangGraph and Tavily that conducts deep diligence on companies using a multi-agent framework. It leverages OpenRouter on the backend for rapid inference with any leading model.
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.
Fast, dependency-free Go package to infer binary file types based on the magic numbers header signature
Bayesian Data Analysis course at Aalto
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
Bayesian inference with probabilistic programming.
Communicate with an LLM provider using a single interface
Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and full-stack tuning.
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.