AirLLM 2.8T LLM inference with single 4GB GPU
- chinese-nlp
- finetune
- generative-ai
- instruct-gpt
132,241 visible repositories in the index
1–5 of 5 · Most stars
AirLLM 2.8T LLM inference with single 4GB GPU
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
A throughput-oriented high-performance serving framework for LLMs
Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA.《大模型算法与系统教程》:模型架构与训练、后训练、推理加速、性能与算子优化,配套 Notebook 练习、答案解析、测试与实验。
🤗 Optimum Intel: Accelerate inference with Intel optimization tools