Repository Index

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

• attention-mechanism
• deep-learning
• gpt
• gpt-2

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

• gpt-2
• adaptation
• language-model
• gpt-3

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

• machine-learning
• natural-language-processing
• tensorflow
• deep-learning

Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.

• language-modeling
• natural-language-generation
• machine-learning
• deep-learning

DBMDZ BERT, DistilBERT, ELECTRA, GPT-2 and ConvBERT models

• bert
• bert-model
• german
• transformers

A minimalistic codebase for training NLG models from HuggingFace Transformers using PyTorch Lightning.

• bart
• generation
• gpt-2
• language-model