Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
Repository Index
47,133 repositories indexed
1–6 of 6 · Most stars
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
OmniEdu: Open Foundation Models for Learning and Teaching
a systemone proxy using embeddings that distills a teacher model
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
- int4
- large-language-models
- pytorch
- quantization
This is the github repository for converting craft pretrained model to tflite version