1.NVIDIA / Megatron-LM
Ongoing research training transformer models at scale
- large-language-models
- model-para
- transformers
+1,051 stars
from 17,272 · score 6.96
Trending repositories
Ranked by Attention Score: stars gained over the last 90 complete days
14 repositories by Attention Score · Aug 6, 2026 – Nov 3, 2026 (UTC)
2.modelscope / ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
+1,011 stars
from 15,019 · score 6.92
3.deepspeedai / DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- deep-learning
- pytorch
- gpu
- machine-learning
- billion-parameters
- data-parallelism
- +7 more
+541 stars
from 42,749 · score 6.30
4.axolotl-ai-cloud / axolotl
Go ahead and axolotl questions
+386 stars
from 12,278 · score 5.96
5.Lightning-AI / pytorch-lightning
Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
+235 stars
from 31,206 · score 5.46
6.Lightning-AI / litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
+186 stars
from 13,558 · score 5.23
7.hpcaitech / ColossalAI
Making large AI models cheaper, faster and more accessible
- deep-learning
- hpc
- large-scale
- data-parallelism
- pipeline-parallelism
- model-parallelism
- +6 more
+116 stars
from 41,316 · score 4.76
8.PaddlePaddle / PaddleNLP
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
- paddlenlp
- pretrained-models
- +13 more
+66 stars
from 12,921 · score 4.20
9.dmlc / dgl
Python package built to ease deep learning on graph, on top of existing DL frameworks.
- deep-learning
- graph-neural-networks
+35 stars
from 14,256 · score 3.58
10.PaddlePaddle / PaddleFormers
PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.
- model
+20 stars
from 12,948 · score 3.04
11.horovod / horovodArchived
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
- tensorflow
- uber
- machine-learning
- machinelearning
- mpi
- baidu
- +7 more
+7 stars
from 14,666 · score 2.08
12.tensorpack / tensorpack
A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility
+6 stars
from 6,278 · score 1.95
13.yahoo / TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
- tensorflow
- spark
- yahoo
- machine-learning
- cluster
- featured
- +2 more
+5 stars
from 3,843 · score 1.79
14.facebookresearch / PyTorch-BigGraphArchived
Generate embeddings from large-scale graph-structured data.
+3 stars
from 3,451 · score 1.39