Repository Index

Distribute and run LLMs with a single file.

• cross-platform
• gguf
• llama-cpp
• local-ai

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

• kv-cache
• llm-inference
• local-llm
• quantization

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

• cuda
• glm
• glm-5
• llm

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

• godot
• godot-engine
• godot-plugin
• godot4

Nornicdb is a distributed low-latency, Graph+Vector, Temporal MVCC with all sub-ms HNSW search, graph traversal, and writes. Using Neo4j Bolt/Cypher and qdrant's gRPC means you can switch with no changes while adding intelligent features like schemas, managed embeddings, reranking+llm, GPU accel, Auto-TLP, Policy-based Memory Decay, and MCP server.

• bolt
• cypher
• database
• enterprise-solutions

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

• anthropic-api
• cuda
• gemma
• gguf

Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

• gguf
• humanizer
• llama-cpp
• llm

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

• apple-silicon
• llm
• llm-inference
• local-llm

An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.

• ai-agent
• ai-assistant
• chatgpt-alternative
• claude-alternative

Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.

• cuda
• deepseek
• deepseek-v4
• gguf

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

• ai
• gguf
• gpu
• inference

TranslatedBig Blue Fish desktop pet with local LLM. A Windows AI desktop pet with a built-in 4B local large model. It uses a whale-girl persona that combines weather, tech news and the current application to keep you gentle company through text bubbles. Supports one-click installation and pausing inference when off power.

• ai-companion
• deepseek
• desktop-companion
• desktop-pet

M-Courtyard: Local AI Model Fine-tuning Assistant for Apple Silicon. Zero-code, zero-cloud, privacy-first desktop app powered by Tauri + React + mlx-lm.

• ai-assistant
• apple-silicon
• desktop-app
• fine-tuning

Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI models that runs completely locally on your computer. No subscriptions, no data sent to the internet — just you and your personal AI assistant

• ai-chat
• artificial-intelligence
• candle
• chatbot

MCP server for Git with local Ollama — zero tokens for git operations

• ai-agents
• claude-code
• conventional-commits
• developer-tools

Translate code comments and strings in Neovim using online or local LLMs and external translators.

• developer-tools
• neovim
• neovim-plugin
• translation

localaik is a local compatibility server for a subset of the Gemini and OpenAI APIs. Run one container locally, point your SDK to it, and the proxy serves both protocol shapes on the same port for tests and development.

• docker
• gemini
• golang
• llama-cpp

Local-LLM alert triage for self-hosted SOCs.

• homelab
• ids
• llm
• ollama