Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
Repository Index
47,133 repositories indexed
1–4 of 4 · Most stars
Infrastructure for the next generation of voice agents, designed to provide universal memory. It is divided into a left brain and a right brain, storing information and emotions respectively, while a fully streaming architecture eliminates latency at the fundamental level.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
Official Inference Code of EAGLE-4
- eagle
- gpu
- llm-inference
- sglang