Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
- artificial-intelligence
- attention-mechanism
- transformers
- computer-vision
99,825 visible repositories in the index
1–24 of 46 · Most stars
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
all kinds of text classification models and more with deep learning
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
A concise but complete full-attention transformer with a set of promising experimental features from various papers
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022; Official code
A collection of important graph embedding, classification and representation learning papers with implementations.
A TensorFlow Implementation of the Transformer: Attention Is All You Need
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
An advanced singing voice synthesis system with high fidelity, expressiveness, controllability and flexibility based on DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
Keras Attention Layer (Luong and Bahdanau scores).
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
High Fidelity Detection Mechanism for RSC/Next.js RCE (CVE-2025-55182 & CVE-2025-66478)
Reformer, the efficient Transformer, in Pytorch
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
Implementation of Alphafold 3 from Google Deepmind in Pytorch
To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released