Ongoing research training transformer models at scale
- large-language-models
- model-para
- transformers
95,842 visible repositories in the index
1–16 of 16 · Most stars
Ongoing research training transformer models at scale
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
Model parallel transformers in JAX and Haiku
Scenic: A Jax Library for Computer Vision Research and Beyond
中文nlp解决方案(大模型、数据、模型、训练、推理)
Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.
Ongoing research training transformer language models at scale, including: BERT & GPT-2
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
Training LLMs with QLoRA + FSDP
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Scalable and memory-optimized training of diffusion models
Proof Of Concept of DEcentralised Machine Learning on top of the Golem (https://golem.network/) architecture