Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
- gpu-sharing
- kvcache
- llm
- sglang
111,085 visible repositories in the index
1–4 of 4 · Most stars
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
An easy to use thread safe in-memory caching service with a simple developer friendly API for c#
The repository host gives no description.
Caching implementation with a variety of storage options, as well as codified caching strategies for callbacks, classes, and output