Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.
- cuda
- desktop-app
- gguf
- llm
94,845 visible repositories in the index
1–1 of 1 · Most stars
Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.