NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Repository Index
47,133 repositories indexed
25–48 of 48 · Most stars
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
- anthropic-api
- cuda
- gemma
- gguf
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- avx2
- c
- cpu-inference
- gguf
Delicious diagnostic debugging in Neovim 🤤
Cleora AI is a general-purpose open-source model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data. Created by Synerise.com team.
- ai
- graphs
- synerise
- embeddings
Tamper-evident tracing, policy gates and cross-checks for AI agents: see what the agent did, and why.
- agent-security
- ai-agents
- audit-log
- claude-code
Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.
Local inference for diffusion language models (LLaDA, Dream), written in Mojo
- cpu
- diffusion-models
- gguf
- inference
OpenAI-compatible async gateway for muse.ai with pluggable drivers (mock / browser-CDP / http)
TranslatedBullFlow — Open-source enterprise RAG solution: knowledge base → automated marketing content → omnichannel publishing. The entire knowledge base pipeline runs on local open-source models, with no paid models needed; AI Agent creates image-text posts / images / videos, with one-click publishing to 17 platforms including Douyin, Xiaohongshu, WeChat Official Accounts, LinkedIn, and YouTube. Commercial-grade, multi-tenant, billing & admin built in. MIT
- ai-agents
- bge-m3
- content-generation
- elasticsearch
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
TranslatedBig Blue Fish desktop pet with local LLM. A Windows AI desktop pet with a built-in 4B local large model. It uses a whale-girl persona that combines weather, tech news and the current application to keep you gentle company through text bubbles. Supports one-click installation and pausing inference when off power.
- ai-companion
- deepseek
- desktop-companion
- desktop-pet
PocketStrike is a privacy-first mobile AI workstation. It combines zero-cloud native on-device GGUF inference (ARM NEON accelerated) │ with frontier cloud LLMs, multi-step ReAct agent tools, Model Context Protocol (MCP) servers, self-evolving cross-session memory, and FLUX neural image │ generation.
Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents
Open standard for Provenance & Intent Contracts (PIC) in AI agents. Verify intent, provenance, and evidence before high-impact tool calls.
- action-proposals
- agentic-ai
- agents
- ai
🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift
The EU & UK compliance plane for Google ADK.
A minimalistic codebase for training NLG models from HuggingFace Transformers using PyTorch Lightning.
- bart
- generation
- gpt-2
- language-model