Ongoing research training transformer models at scale
- large-language-models
- model-para
- transformers
107,098 visible repositories in the index
1–20 of 20 · Most stars
Ongoing research training transformer models at scale
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
Model parallel transformers in JAX and Haiku
Translatedcube studio is an open-source, cloud-native, one-stop machine learning / deep learning / large model AI platform. It covers the full mlops algorithm pipeline, a computing power rental platform, online notebook development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference services with VGPU virtualization, edge computing, a labeling platform with automated labeling, SFT fine-tuning / reward model / reinforcement learning training for large models such as deepseek, multi-machine large model inference with vllm/ollama/mindie, a private knowledge base, and an AI model marketplace. It supports domestic Chinese CPUs/GPUs/NPUs and the Ascend ecosystem, supports RDMA, and supports distributed frameworks such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano.
Scenic: A Jax Library for Computer Vision Research and Beyond
中文nlp解决方案(大模型、数据、模型、训练、推理)
Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台,算法全链路流程,多租户,算力租赁平台,token中转,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务,VGPU虚拟化,云边端协同,边缘计算,自动化标注平台,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库llmops智能体,AI模型市场,支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等,支持ib/roce/RDMA,信创支持
Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.
A simple, performant, and scalable Jax LLM!
Ongoing research training transformer language models at scale, including: BERT & GPT-2
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Scalable and memory-optimized training of diffusion models
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch