Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Repository Index
99,825 visible repositories in the index
1–15 of 15 · Most stars
DeepEP: an efficient expert-parallel communication library
A Next-Generation Training Engine Built for Ultra-Large MoE Models
EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL
Translatedcube studio is an open-source, cloud-native, one-stop machine learning / deep learning / large model AI platform. It covers the full mlops algorithm pipeline, a computing power rental platform, online notebook development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference services with VGPU virtualization, edge computing, a labeling platform with automated labeling, SFT fine-tuning / reward model / reinforcement learning training for large models such as deepseek, multi-machine large model inference with vllm/ollama/mindie, a private knowledge base, and an AI model marketplace. It supports domestic Chinese CPUs/GPUs/NPUs and the Ascend ecosystem, supports RDMA, and supports distributed frameworks such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano.
The central nervous system and backend orchestrator for Fractal. It handles multi-tenant task scheduling, real-time TFLOPs budgeting, and federated weight aggregation using FedAvg.
Massively Parallel Deep Reinforcement Learning. 🔥
- pytorch
- reinforcement-learning
- ppo
- sac
Open-source framework for the research and development of foundation models.
A bidirectional pipeline parallelism algorithm for computation-communication overlap in DeepSeek V3/R1 training.
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
- ai
- allreduce
- amd
- broadcom
Muon is Scalable for LLM Training
Multimodal RL training framework for diffusion & omni models
- diffusion-models
- flow-matching
- grpo
- multimodal
Collective communications library with various primitives for multi-machine training.
- collectives
- distributed-training
- pytorch
A fast communication-overlapping library for tensor/expert parallelism on GPUs.
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
- huggingface-transformers
- language-vision
- multimodal
- pytorch