Repository Index

💫 Industrial-strength Natural Language Processing (NLP) in Python

• natural-language-processing
• data-science
• machine-learning
• python

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Pretrained models for TensorFlow.js

A very simple framework for state-of-the-art Natural Language Processing (NLP)

• pytorch
• nlp
• named-entity-recognition
• sequence-labeling

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

• nlp
• embedding
• bert
• ernie

modest natural-language processing

• nlp
• part-of-speech
• named-entity-recognition

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

• python
• pii
• privacy
• data-anonymization

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more

• nlp
• nlu
• nodejs
• sentiment-analysis

Naive linter for English prose

VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.

Easily train your own text-generating neural network of any size and complexity on any text dataset with a few lines of code.

• keras
• deep-learning
• text-generation
• tensorflow

Seamlessly integrate LLMs into scikit-learn.

• chatgpt
• deep-learning
• llm
• machine-learning

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

• machine-learning
• natural-language-processing
• tensorflow
• deep-learning

Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining

A full spaCy pipeline and models for scientific/biomedical documents.

• scientific-documents
• spacy
• custom-pipes
• nlp

Self-contained Machine Learning and Natural Language Processing library in Go

• deep-learning
• machine-learning
• natural-language-processing
• neural-network

Pretrained model hub for Keras 3.

• deep-learning
• keras
• machine-learning
• nlp

Self-contained Japanese Morphological Analyzer written in pure Go

• japanese
• tokenizer
• nlp-library
• japanese-language

The Classical Language Toolkit

• nlp
• ai
• python
• nltk

🚀 Catalyst is a C# Natural Language Processing library built for speed. Inspired by spaCy's design, it brings pre-trained models, out-of-the box support for training word and document embeddings, and flexible entity recognition models.

• nlp
• natural-language-processing
• artificial-intelligence
• embeddings

Parse human names into title, given, middle, family, suffix, nickname and maiden name. Typed, zero-dependency, CJK-aware.

• python
• text-processing
• text-parser
• python-module

RNNLG is an open source benchmark toolkit for Natural Language Generation (NLG) in spoken dialogue system application domains. It is released by Tsung-Hsien (Shawn) Wen from Cambridge Dialogue Systems Group under Apache License 2.0.

• natural-language-processing
• natural-language-generation
• nlg-dataset
• deep-learning