“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。
Repository Index
111,085 visible repositories in the index
145–168 of 192 · Most stars
MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.
- llm
- lmstudio
- macbook
- mlx
A throughput-oriented high-performance serving framework for LLMs
Awesome Machine Unlearning (A Survey of Machine Unlearning)
- continual-learning
- data-deletion
- data-privacy
- data-removal
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
- cpu
- cpu-inference
- deep-learning
- faiss
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
List of awesome hosting sorted by minimal plan price
- nodejs
- hosting
- vps
- free
A scalable inference server for models optimized with OpenVINO™
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
- speculative-decoding
The Triton TensorRT-LLM Backend
LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving
- sig-apps
- llm-inference
A highly optimized LLM inference acceleration engine for Llama and its variants.
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
- llm
- llm-inference
- triton-kernels
- vllm
Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.
- android
- android-17
- edge-ai
- jetpack-compose
List of papers related to neural network quantization in recent AI conferences and journals.
- quantization
- awesome-list
- papers
- neural-networks
The most atomic way to train and inference a GPT in pure, dependency-free C
- artificial-intelligence
- c
- c-programming
- deep-learning
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
- apple-silicon
- clover
- coding-agents
- decision-models
The repository host gives no description.
Ontology-Augmented Generation for AI agents.
- agent-skills
- document-parsing
- knowledge-graph
- mineru
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- avx2
- c
- cpu-inference
- gguf
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
Local inference for diffusion language models (LLaDA, Dream), written in Mojo
- cpu
- diffusion-models
- gguf
- inference
🔥🔥 The extremely fast inference engine for decision model
- gemma
- inference-engine
- llm-inference
- llm-serving