An open, programmable decision layer for models and compute.
- mixture-of-models
- semantic-router
- vllm
- ai-gateway
107,098 visible repositories in the index
1–7 of 7 · Most stars
An open, programmable decision layer for models and compute.
Manages Unified Access to Generative AI Services built on Envoy Gateway
Router that virtually distributes inference across connected devices in the home.
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
csghub-server is the backend server for CSGHub which helps user to manage datasets, modes, and also run Model Inference, Finetune and Application Spaces.
Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off.
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.