1.NVIDIA / Megatron-LMOngoing research training transformer models at scalelarge-language-modelsmodel-paratransformersPythonOther license18K total4.7K forks+36 starsfrom 18,291 · score 3.61
2.horovod / horovodArchivedDistributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.tensorflowubermachine-learningmachinelearningmpibaidu+7 morePythonOther license15K total2.2K forks+1 starsfrom 14,674 · score 0.69