Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Repository Index
94,845 visible repositories in the index
1–6 of 6 · Most stars
A framework for efficient model inference with omni-modality models
- diffusion
- inference
- model-serving
- pytorch
TranslatedOffline translation model server with low resource consumption, fast speed, and private deployment capability.
The repository host gives no description.
- docker
- docker-compose
- faster-whisper
- openai-api
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
- nlp
- transformer
- bert
- decoder