LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Repository Index
108,094 visible repositories in the index
1–24 of 31 · Most stars
☁️ Build multimodal AI applications with cloud-native stack
- neural-search
- cloud-native
- deep-learning
- machine-learning
🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
- bert
- sentence-encoding
- deep-learning
- clip-model
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Jev-like family of decision models built on top of Qwen3.5/3.8 you can train and run on your own
- decision-model
- jev
- qwen3
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Production infrastructure for machine learning at scale
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
- machine-learning
- scale
- workflow
- mlops
A framework for efficient model inference with omni-modality models
- diffusion
- inference
- model-serving
- pytorch
ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
- version-control
- experiment-manager
- version
- control
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
- knative
- machine-learning
- model-interpretability
- model-serving
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
TranslatedOffline translation model server with low resource consumption, fast speed, and private deployment capability.
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
- federated-learning
- deep-learning
- distributed-training
- edge-ai
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
- ai
- api
- serving
- artificial-intelligence
The repository host gives no description.
- docker
- docker-compose
- faster-whisper
- openai-api
Open-source inference server and production cluster for all the models your agent needs.
🪄 Turns your machine learning code into microservices with web API, interactive GUI, and more.
- fastapi
- streamlit
- pydantic
- python
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.
- machine-learning
- python
- gpu
- llms
High Performance Serving Runtime For Interactive Super Intelligence System
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
- mlops
- python
- data-science
- machine-learning
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
The backend agents build with - Multimodal database, orchestration, and serving in one file
a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
- nlp
- transformer
- bert
- decoder