Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
- gpt
- large-language-models
- llm
- python
99,825 visible repositories in the index
1–24 of 101 · Most stars
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
An unidentifiable mechanism that helps you bypass GFW.
TranslatedQiu Xipeng's "Neural Networks and Deep Learning" (神经网络与深度学习), Second Edition and General Edition: e-book, chapter table of contents, learning resources, and errata.
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
Decentralized Autonomous Regulated Company (DARC), a company virtual machine that runs on any EVM-compatible blockchain, with on-chain law system, multi-level tokens and dividends mechanism.
TranslatedA Trojan proxy written in Go. An unidentifiable mechanism that helps you bypass GFW. Supports multiplexing, routing, CDN relay, and Shadowsocks obfuscation plugins; multi-platform, no dependencies. https://p4gefau1t.github.io/trojan-go/
all kinds of text classification models and more with deep learning
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
Provides mechanisms for walking through any arbitrary PHP variable
A concise but complete full-attention transformer with a set of promising experimental features from various papers
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022; Official code
A collection of important graph embedding, classification and representation learning papers with implementations.
RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.
A TensorFlow Implementation of the Transformer: Attention Is All You Need
Build a modern LLM from scratch. Every line commented. Explained like we are five.
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
A denoising autoencoder + adversarial losses and attention mechanisms for face swapping.
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
An advanced singing voice synthesis system with high fidelity, expressiveness, controllability and flexibility based on DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
the LIBpcap interface to various kernel packet capture mechanism