A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Repository Index
47,133 repositories indexed
1–24 of 26 · Most stars
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
- attention-mechanism
- deep-learning
- gpt
- gpt-2
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
- accelerator
- ai-infra
- ai-infrastructure
- book
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
- python
- utilities
- cache
- data-structures
Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
- python
- cache
- persistence
- key-value-store
Compresses linked and inline javascript or CSS into a single cached file.
Extensible memoizing collections and decorators
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
A slick ORM cache with automatic granular event-driven invalidation.
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
Asyncio cache manager for redis, memcached and memory
知らない分野を、原文にたどりつくまで芋づる式に掘る Claude Code のスキル
30 different honeypots in one package! (dhcp, dns, elastic, ftp, http proxy, https proxy, http, https, imap, ipp, irc, ldap, memcache, mssql, mysql, ntp, oracle, pjl, pop3, postgres, rdp, redis, sip, smb, smtp, snmp, socks5, ssh, telnet, vnc)
- pypi
- monitoring-tool
- network-analysis
- malware-analysis
YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
Lightweight Claude Code statusLine: 5h/7d rate-limit usage, reset countdowns, model + context window, prompt-cache age — one line, 3 styles × 9 themes, daemon fast-mode
dogpile.cache is a Python caching API which provides a generic interface to caching backends of any variety
"Evolving AppCompat/AmCache data analysis beyond grep"
Official implementation of paper "Cache Engineering for Agents"
🐍 CLI tool to find and delete unused Python virtual environments (.venv, poetry, conda, pipenv, uv). Reclaim disk space effortlessly — supports bulk delete, size reporting, and pycache cleanup.
- command-line
- developer-tools
- python
- venv
A simple page cache for Wagtail based on the Django cache middleware.
Parses Java Cache IDX files
micropython SD card driver that works with LittleFS2 (implements extended interface)