Repository Index

LLM Frontend for Power Users.

• ai
• chat
• llm

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.

• nextjs
• openai
• llm
• rag

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

• apple-silicon
• inference-server
• llm
• macos

TranslatedProduction-grade platform for building agentic IM bots - Agent, knowledge base orchestration, plugin system / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信 (Qiye Weixin, "Enterprise WeChat"), 企微智能机器人 (Qiwei Zhineng Jiqiren, "Enterprise WeChat smart bot"), 公众号 (Gongzhonghao, "official account")) / 飞书 (Feishu, "flying book") / 钉钉 (Dingding, "nail nail") / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow

• qq
• openai
• agent
• llm

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

• dpo
• fine-tuning
• inference
• llama

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

• agents
• guardrails
• python
• safety

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

• gemma
• gpgpu
• llm
• local-ai

The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.

• apple-intelligence
• apple-silicon
• cli
• foundationmodels

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

• llava
• llm
• mlx
• vision-transformer

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

• agent
• anthropic-api
• apple-silicon
• claude-code

A local inference engine for Apple silicon, built around the model.

• apple-silicon
• coding-agents
• llm-inference
• macos

TranslatedAn Agent tavern based on DSH. A tavern-style text game Agent. Compatible with the SillyTavern ecosystem; character cards can be imported directly and played. Faster, more stable, more lively. Works with all models. Can also be played on mobile phones.

• dsh-plugin
• sillytavern
• agent
• role-play

🐋 Run 284B DeepSeek-V4-Flash (<59GiB RAM, 44.4 tok/s pp, 9.1-14.7 tok/s) & 30B MoEs (25-111 tok/s) on a pure 64GB CPU at 256K ctx (0 GPUs). SwiGLU-H2, Stockfish-PVS L3 & 13ms Claude Code restore.

• avx512
• cpu-inference
• hpc
• llama-cpp

Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai

• apple-silicon
• clover
• coding-agents
• decision-models

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

• apple-silicon
• llm
• llm-inference
• local-llm

A large language model (LLM) plugin for Neovim, provides commands to interact with LLM (like ChatGPT, Copilot, ChatGLM, kimi, deepseek, openrouter and local llms). Support Github models.

• chatgpt
• github-models
• deepseek
• ollama

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• apple-silicon
• cuda
• gguf
• inference

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

• ai
• gguf
• gpu
• inference

A DeepSeek Harness (dsh) plugin — turns dsh web into a SillyTavern-style roleplay frontend: character cards, prompt presets, lorebooks, personas, BM25 long-term memory, world-state deltas, and rollback-able floor operations, all on the dsh agent runtime.

• dsh-plugin
• deepseek-harness
• deepseek-harness-plugin
• sillytavern

M-Courtyard: Local AI Model Fine-tuning Assistant for Apple Silicon. Zero-code, zero-cloud, privacy-first desktop app powered by Tauri + React + mlx-lm.

• ai-assistant
• apple-silicon
• desktop-app
• fine-tuning

Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app.

• ai
• artificial-intelligence
• coding-agent
• llm

PDF extraction that audits its own output — and certifies any other extractor's, catching pages they silently dropped. Verify signed manifests offline: free, MIT, no account. 0.903 on opendataloader-bench, #2 of 8 engines. 7-tool MCP server.

• llm
• mcp
• ocr
• pdf

Pipeline de ingestão e busca semântica de documentos para RAG — com rate limiting, circuit breaker e fila resiliente.

• docker
• pipeline
• python
• rag