TranslatedProvides a practical interactive interface for large language models (LLMs) such as GPT/GLM, with special optimization for the paper reading/polishing/writing experience. Modular design, supports custom shortcut buttons & function plugins, supports project analysis & self-translation for Python, C++ and other projects, PDF/LaTeX paper translation & summarization, supports querying multiple LLM models in parallel, and supports local models such as chatglm3. Integrates Tongyi Qianwen (Qwen), deepseekcoder, iFlytek Spark, ERNIE Bot (Wenxin Yiyan), llama2, rwkv, claude2, moss, and more.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Topic Modelling for Humans
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Official implementation of paper "Cache Engineering for Agents"
Run scripts, explore your genome, and get personalized explanations — all in natural dialogue.