💫 Industrial-strength Natural Language Processing (NLP) in Python
Repository Index
47,133 repositories indexed
1–3 of 3 · Most stars
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages