Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Repository Index
47,133 repositories indexed
1–24 of 29 · Most stars
The simplest, fastest repository for training/finetuning medium-sized GPTs.
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
[EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation
- knowledge-graph
- large-language-models
- retrieval-augmented-generation
- genai
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
TranslatedLangchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain
- chatglm
- langchain
- llm
- knowledge-base
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
- attention-mechanism
- deep-learning
- gpt
- gpt-2
[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.
A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
TranslatedTencent CodeBuddy account pool management console with a built-in OpenAI-compatible reverse proxy gateway: bulk onboarding of accounts via QR code scan, scheduled check-ins and keep-alive, key grouping and distribution, IP and model whitelists, call logs and usage statistics; supports one-click updates with signature verification.
- account-manager
- api-gateway
- codebuddy
- fastapi
1M param OCR model - runs on CPU, 15x faster than tesseract, close to 1B+ param models in accuracy
Source-available agent platform for Global × China enterprises — wire every system through one agent core. Self-hosted, any LLM.
A framework to build Slack apps using Python
- slack
- slack-api
- bolt
- web-api
MCP-NixOS - Model Context Protocol Server for NixOS resources
- ai-assistant
- ai-integration
- ai-tools
- anthropic
Run many AI agents at once on your Mac, up to 3x faster.
- agents
- apple-silicon
- artificial-intelligence
- inference-acceleration
Lightweight Claude Code statusLine: 5h/7d rate-limit usage, reset countdowns, model + context window, prompt-cache age — one line, 3 styles × 9 themes, daemon fast-mode
和《偶像大师 闪耀色彩》偶像实时语音/文字对话的 AI 分身：日语原声 + 中文字幕、回忆原作剧情、跨角色共享记忆；支持 Google/GitHub 登录与自带密钥，Docker Compose 一键部署（参考 LAPLACE Eidolon）
- ai-companion
- character-ai
- fastapi
- idolmaster
Our lightning-fast inference engine.
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents
Friendly Environment for Neural Networks (fenn) is a simple Python framework for building ML/DL workflows and LLM agents faster, with prebuilt trainers, agent templates, logging, configuration management, and much more.
- beginner-friendly
- beginner-project
- deep-learning
- deep-neural-networks