LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Repository Index
96,838 visible repositories in the index
1–24 of 35 · Most stars
TranslatedLLM API management & key redistribution system, unifying multiple providers under a single API. Single binary, Docker-ready, with an English UI. Supports mainstream models such as OpenAI, Azure, Anthropic Claude, Google Gemini, DeepSeek, 字节豆包 (Doubao, "bean bun", from ByteDance), ChatGLM, 文心一言 (Wenxin Yiyan, "literary heart, one utterance"), 讯飞星火 (Xunfei Xinghuo, "iFlytek Spark"), 通义千问 (Tongyi Qianwen, "Tongyi thousand questions"), 360 智脑 (360 Zhinao, "360 smart brain") and 腾讯混元 (Tencent Hunyuan, "primordial chaos"), with unified API adaptation, usable for key management and secondary distribution. One-click deployment, ready to use out of the box.
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
The Cloud Native AI Platform
- ml
- kubernetes
- minikube
- tensorflow
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
Generates original ARC-AGI-1-style tasks distribution-matched to the public eval set.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
A Datacenter Scale Distributed Inference Serving Framework
- diffusion
- disaggregated-serving
- kubernetes
- llm-inference
TranslatedOpen-source manuscript of 《深入理解 AI Infra:量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Optimize prompts, code, and more with AI-powered Reflective Optimization
- agent-skills
- coding-agent
- discovery
- distributed-evolutionary-algorithms
Software in C and data files for the popular GloVe model for distributed word representations, a.k.a. word vectors or embeddings
Democratizing Reinforcement Learning for LLMs
- agent-framework
- agentic-workflow
- distributed-training
- llm-training
State of the art inference performance with modern accelerators on Kubernetes
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.
- agents
- ai
- decentralized
- distributed
Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系,成为中文AIGC和认知智能的基础设施。
- chinese-nlp
- pretrained-models
- pytorch
- distributed-training
Distributed inference for mobile, desktop and server.
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
IoT Platform Framework
- golang
- iot-platform
- edge-computing
- distributed-systems
Machine learning metrics for distributed, scalable PyTorch applications.
- python
- data-science
- machine-learning
- pytorch
Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.
VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
Distributed AI Model Training and LLM Fine-Tuning on Kubernetes