1.NVIDIA / Megatron-LM
Ongoing research training transformer models at scale
- large-language-models
- model-para
- transformers
+1,051 stars
from 17,272 · score 6.96
Trending repositories
Ranked by Attention Score: stars gained over the last 90 complete days
15 repositories by Attention Score · Aug 6, 2026 – Nov 3, 2026 (UTC)
2.modelscope / ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
+1,011 stars
from 15,019 · score 6.92
3.microsoft / onnxruntime
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
- deep-learning
- onnx
- neural-networks
- machine-learning
- ai-framework
- hardware-acceleration
- +3 more
+1,003 stars
from 21,215 · score 6.91
4.deepspeedai / DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- deep-learning
- pytorch
- gpu
- machine-learning
- billion-parameters
- data-parallelism
- +7 more
+541 stars
from 42,749 · score 6.30
5.axolotl-ai-cloud / axolotl
Go ahead and axolotl questions
+386 stars
from 12,278 · score 5.96
6.Lightning-AI / pytorch-lightning
Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
+235 stars
from 31,206 · score 5.46
7.Lightning-AI / litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
+186 stars
from 13,558 · score 5.23
8.PaddlePaddle / Paddle
TranslatedPArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (Core framework of 'PaddlePaddle' (飞桨, Feijiang), for deep learning & machine learning: high-performance single-machine and distributed training, and cross-platform deployment)
- paddlepaddle
- deep-learning
- scalability
- machine-learning
- neural-network
- python
- +2 more
+157 stars
from 23,977 · score 5.06
9.hpcaitech / ColossalAI
Making large AI models cheaper, faster and more accessible
- deep-learning
- hpc
- large-scale
- data-parallelism
- pipeline-parallelism
- model-parallelism
- +6 more
+116 stars
from 41,316 · score 4.76
10.PaddlePaddle / PaddleNLP
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
- paddlenlp
- pretrained-models
- +13 more
+66 stars
from 12,921 · score 4.20
11.dmlc / dgl
Python package built to ease deep learning on graph, on top of existing DL frameworks.
- deep-learning
- graph-neural-networks
+35 stars
from 14,256 · score 3.58
12.microsoft / CNTKArchived
Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
- cognitive-toolkit
- cntk
- deep-learning
- machine-learning
- deep-neural-networks
- neural-network
- +5 more
+26 stars
from 17,557 · score 3.30
13.PaddlePaddle / PaddleFormers
PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.
- model
+20 stars
from 12,948 · score 3.04
14.horovod / horovodArchived
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
- tensorflow
- uber
- machine-learning
- machinelearning
- mpi
- baidu
- +7 more
+7 stars
from 14,666 · score 2.08
15.apache / mxnetArchived
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
- mxnet
+2 stars
from 20,791 · score 1.10