Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Repository Index
55,140 visible repositories in the index
1–5 of 5 · Most stars
Neovim edit completion plugin
- nvim
- cursor-prediction
- edit-completion
Local LLM API Intelligent Gateway — Open-source client + Server Open Core model, unified management of multiple LLM Provider channels, providing intelligent routing, session stickiness, cost optimization, full-chain observability, and team collaborative governance.
Manage and use multiple Ollama instances with automatic offline detection/failover and model availability tracking
A Go toolkit for building AI agents and applications across multiple providers. Unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking.
- ai
- golang
- llm
- structured-output