LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Repository Index
76,855 visible repositories in the index
1–11 of 11 · Most stars
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
Computing with Python functions.
- python
- parallel-computing
- caching
- multiprocessing
Full featured redis cache backend for Django.
Extensible memoizing collections and decorators
A slick ORM cache with automatic granular event-driven invalidation.
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
- gpu-sharing
- kvcache
- llm
- sglang
fastapi-cache is a tool to cache fastapi response and function result, with backends support redis and memcached.
Asyncio cache manager for redis, memcached and memory
dogpile.cache is a Python caching API which provides a generic interface to caching backends of any variety
In-memory key-value server in Rust that speaks RESP, so redis-cli and Redis clients connect. Locks, rate limits, job queues and a change feed are single commands; Raft-based failover. Clients for Rust, .NET, Go, Java, Python and Node.js.