AirLLM 2.8T LLM inference with single 4GB GPU
- chinese-nlp
- finetune
- generative-ai
- instruct-gpt
111,085 visible repositories in the index
1–5 of 5 · Most stars
AirLLM 2.8T LLM inference with single 4GB GPU
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Distill your LLM traces into a fast hybrid System 1 models. Open-source ML + LLM inference with calibrated deferral.
A throughput-oriented high-performance serving framework for LLMs
Awesome Machine Unlearning (A Survey of Machine Unlearning)