LLM Frontend for Power Users.
Repository Index
47,133 repositories indexed
1–5 of 5 · Most stars
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
TranslatedLangGPT: Empowering everyone to become a prompt expert! 🚀 📌 Proposer of Structured Prompt 📌 Initiator of Meta-Prompt 📌 The most popular paradigm for putting prompts into practice | Language of GPT The pioneering framework for structured & meta-prompt design 10,000+ ⭐ | Battle-tested by thousands of users worldwide Created by 云中江树 (Yunzhong Jiangshu, "river and trees amid the clouds")
The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2
- amd
- llm
- strix-halo
- gfx1151
ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenClaw and custom harnesses.