Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
- artificial-intelligence
- attention-mechanism
- transformers
- computer-vision
99,825 visible repositories in the index
1–24 of 42 · Most stars
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
all kinds of text classification models and more with deep learning
A concise but complete full-attention transformer with a set of promising experimental features from various papers
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022; Official code
A TensorFlow Implementation of the Transformer: Attention Is All You Need
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
A denoising autoencoder + adversarial losses and attention mechanisms for face swapping.
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
An advanced singing voice synthesis system with high fidelity, expressiveness, controllability and flexibility based on DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
Keras Attention Layer (Luong and Bahdanau scores).
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
Reformer, the efficient Transformer, in Pytorch
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
Implementation of Alphafold 3 from Google Deepmind in Pytorch
To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
Bi-directional Attention Flow (BiDAF) network is a multi-stage hierarchical process that represents context at different levels of granularity and uses a bi-directional attention flow mechanism to achieve a query-aware context representation without early summarization.