TranslatedProvides a practical interactive interface for large language models (LLMs) such as GPT/GLM, with special optimization for the paper reading/polishing/writing experience. Modular design, supports custom shortcut buttons & function plugins, supports project analysis & self-translation for Python, C++ and other projects, PDF/LaTeX paper translation & summarization, supports querying multiple LLM models in parallel, and supports local models such as chatglm3. Integrates Tongyi Qianwen (Qwen), deepseekcoder, iFlytek Spark, ERNIE Bot (Wenxin Yiyan), llama2, rwkv, claude2, moss, and more.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Topic Modelling for Humans
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Utils for streaming large files (S3, HDFS, gzip, bz2...)
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
One keyboard and mouse for a Mac and a Windows PC over the local network, in either direction.
- clipboard-sync
- keyboard-sharing
- macos
- mouse-sharing
Credentials catching honeypot
Official implementation of paper "Cache Engineering for Agents"
Assigns priority scores to audio and subtitle tracks in MKV files
Run scripts, explore your genome, and get personalized explanations — all in natural dialogue.
An Easy To Use PyTorch Computer Vision Library
Normalizer for honeypot data.
A Tool to Summarize Web Archive Holdings
Dual listbox for Dash. Original component: https://rawgit.com/jyotirmaybanerjee/react-duallist/master/example/examples.html#
React-color in Dash Original component: https://casesandberg.github.io/react-color/
micropython class for Si7005 relative humidity/temperature sensor