DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Repository Index
111,820 visible repositories in the index
1–15 of 15 · Most stars
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
- deepspeed-library
- gpt-3
- transformers
- language-model
A PyTorch native platform for training generative AI models
PyTorch native post-training library
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
Massively Parallel Deep Reinforcement Learning. 🔥
- pytorch
- reinforcement-learning
- ppo
- sac
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
- federated-learning
- deep-learning
- distributed-training
- edge-ai
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
- deep-learning
- pytorch
- volunteer-computing
- mixture-of-experts
The first distributed AGI system. Thousands of autonomous AI agents collaboratively train models, share experiments via P2P gossip, and push breakthroughs here. Fully peer-to-peer. Join from your browser or CLI.
- agi
- ai-agents
- ai-research
- artificial-general-intelligence
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
- ai
- allreduce
- amd
- broadcom
Collective communications library with various primitives for multi-machine training.
- collectives
- distributed-training
- pytorch
A scalable, agentic-first, and HuggingFace-native RL framework for research (9k lines).
- agentic-rl
- expert-parallelism
- fsdp
- llm
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
- huggingface-transformers
- language-vision
- multimodal
- pytorch
🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
- llm
- vlm
- finetuning
- gemma3