Repository Index

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

• attention-mechanism
• deep-learning
• gpt
• gpt-2

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

• gpt-2
• adaptation
• language-model
• gpt-3

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

• machine-learning
• natural-language-processing
• tensorflow
• deep-learning

This Word Does Not Exist

• machine-learning
• gpt-2
• transformers
• natural-language-processing

零基础入门大模型（LLM）的基础教程：从零手写 GPT-2，一行行看懂大模型是怎么工作的 | Build GPT-2 from scratch in PyTorch, step by step

• attention
• chinese
• deep-learning
• from-scratch

A minimalistic codebase for training NLG models from HuggingFace Transformers using PyTorch Lightning.

• bart
• generation
• gpt-2
• language-model