💫 Industrial-strength Natural Language Processing (NLP) in Python
Repository Index
47,133 repositories indexed
1–5 of 5 · Most stars
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
- natural-language-processing
- nlp
- nlp-parsing
- named-entity-recognition
Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages
Simple and customizable text tokenization gem.
- rubynlp
- ruby
- nlp
- tokenization