1.deepspeedai / DeepSpeedDeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.deep-learningpytorchgpumachine-learningbillion-parametersdata-parallelism+7 morePythonAPACHE-2.043K total5.1K forks+541 starsfrom 42,740 · score 6.30
2.hpcaitech / ColossalAIMaking large AI models cheaper, faster and more accessibledeep-learninghpclarge-scaledata-parallelismpipeline-parallelismmodel-parallelism+6 morePythonAPACHE-2.041K total4.5K forks+121 starsfrom 41,312 · score 4.80