Tags / Machine learning /
Distributed training
A library that trains models across many GPUs or machines, with data, model, or pipeline parallelism.
Repositories
17
Net stars gained, last 7 days
+283
Top-level Tag
Net stars gained each day, last 90 days
Largest
Most stars in total now.
description:
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
description:
Making large AI models cheaper, faster and more accessible
readme:
Lightning handles the engineering, and scales from CPU to multi-node GPUs without changing your core code.
readme:
PaddlePaddle automatically discovers the most efficient distributed parallel strategy.
readme:
**ONNX Runtime training** can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts.
- apache/mxnetarchived · 21K
readme:
Scales up to multi GPUs and distributed setting with auto parallelism through ps-lite, Horovod, and BytePS.
readme:
**Parallelism Strategies** - Scale training across GPUs with TP, PP, DP, EP, and CP
- microsoft/CNTKarchived · 18K
readme:
It implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers.
readme:
Supports distributed data parallelism (DDP), device_map simple model parallelism, DeepSpeed ZeRO2 ZeRO3, FSDP/FSDP2, and Megatron distributed training technologies.
- horovod/horovodarchived · 15K
description:
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
Newest Admissions
Entered the index by the rising rule, newest first.
readme:
Fine-grained GPU allocation built on Ray Placement Groups lets you switch between intra-node multiplexing and disaggregated deployment with a single flag
How a repository gets this Tag
An agent reads the description, the GitHub topics, and the start of the README of each repository. It chooses Tags only from the Tag list in the AwesomeRepos source code. For each Tag, it quotes the text. A code check keeps the Tag only when the quote is in that text. The quote under each repository is that evidence. A Tag does not rank repositories. It does not say that one repository can replace another. To report a wrong Tag, use "Report a problem with this page" on the page of the repository. All numbers on this page count only visible repositories.