A collection of libraries to optimise AI model performances
Repository Index
132,241 visible repositories in the index
121–144 of 1,100 · Most stars
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
- cuda-kernels
- deepspeed
- fastertransformer
- llm-inference
TranslatedOpen-source manuscript of 《深入理解 AI Infra:量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Stylish, intuitive and user-friendly prompts. Used by eslint, webpack, yarn, pm2, pnpm, RedwoodJS, FactorJS, salesforce, Cypress, Google Lighthouse, Generate, tencent cloudbase, lint-staged, gluegun, hygen, hardhat, AWS Amplify, GitHub Actions Toolkit, @airbnb/nimbus, and more! Please follow Enquirer's author: https://github.com/jonschlinkert
- prompt
- inquirer
- cli
- readline
Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
- ingestion-api
- ocr
- omniparser
- parse-server
Minimal web UI for ChatGPT.
Fast, flexible LLM inference
- llm
- rust
- uqff
TranslatedSelf-hosted knowledge agent platform for RAG, knowledge graphs and multi-agent workflows. Yuxi = Cloud Agents + Knowledge RAG, Self-hosted knowledge agent platform for RAG, knowledge graphs and multi-agent workflows.
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
- evaluation
- benchmark
- large-language-model
- llm
ClawX is a desktop app that provides a graphical interface for OpenClaw AI agents. It turns CLI-based AI orchestration into a desktop experience without using the terminal. China website is https://clawx.com.cn.
A pixel office for your OpenClaw: turn invisible work states into a cozy little space with characters, daily notes, and guest agents. Code under MIT; art assets for non-commercial learning only.
- agent-collaboration
- ai-assistant
- dashboard
- flask
A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
This repo powers my experiment where ChatGPT manages a real-money micro-cap stock portfolio.
Optimize prompts, code, and more with AI-powered Reflective Optimization
- agent-skills
- coding-agent
- discovery
- distributed-evolutionary-algorithms
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
- gemma
- gpgpu
- llm
- local-ai
File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.
Framework for evaluating and improving agents
- evals
- rl-environments
- terminal-bench
TranslatedSelf-hosted AI gateway for multi-channel, multi-credential setups — API keys and subscription accounts, scheduling, failover, request logs and usage.
The AI workspace for experts. Switch across 800+ models mid-chat, tune 40+ parameters, Beam models side by side. BYO keys, self-hostable. Built on our native-protocol engine: reliable, hyper-configurable, and fast.
- ai-workspace
- multi-model
- byok
- llm
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it.
Play ChatGPT and other LLM with Xiaomi AI Speaker
✨ AI Coding, Vim Style
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.