Repository Index

Unsupervised text tokenizer for Neural Network-based text generation.

• neural-machine-translation
• natural-language-processing
• word-segmentation

CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

• natural-language-processing
• nlp
• nlp-parsing
• named-entity-recognition

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

• python
• nlp
• natural-language-processing
• machine-learning

TranslatedThe pkuseg toolkit for multi-domain Chinese word segmentation

• chinese-word-segmentation

Data augmentation for NLP

• nlp
• augmentation
• machine-learning
• artificial-intelligence

百度NLP：分词，词性标注，命名实体识别，词重要性

• word-segmentation
• part-of-speech-tagger
• named-entity-recognition
• chinese-word-segmentation

SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

• levenshtein
• fuzzy-search
• approximate-string-matching
• edit-distance

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

• machine-learning
• security
• natural-language-processing
• nlp

Jiagu深度学习自然语言处理工具 知识图谱关系抽取 中文分词 词性标注 命名实体识别 情感分析 新词发现 关键词 文本摘要 文本聚类

• nlp
• cws
• pos
• ner

中文分词

• hanlp
• natural-language-processing
• chinese-word-segmentation
• part-of-speech-tagger

Go efficient fast multilingual NLP and text segmentation; support English, Chinese, Japanese and others.

• go
• segment
• nlp
• gse

文本挖掘和预处理工具（文本清洗、新词发现、情感分析、实体识别链接、关键词抽取、知识抽取、句法分析等），无监督或弱监督方法

• nlp
• sentiment-analysis
• new-word-discovery
• unsupervised

Unsupervised Word Segmentation for Neural Machine Translation and Text Generation

• neural-machine-translation
• segmentation
• machine-translation
• nmt

Deep Learning Chinese Word Segment

• nlp
• deep-learning
• chinese-text-segmentation
• tensorflow

NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.

• pytorch
• ner
• sequence-labeling
• crf

一键中文数据增强包 ； NLP数据增强、bert数据增强、EDA：pip install nlpcda

• nlp
• data-augmentation
• chinese-data-augmentation
• nlpcda

Translated"Jieba" (Chinese for "to stutter") Chinese text segmentation: built to be the best PHP Chinese word segmentation module.

• nlp
• natural-language-processing
• chinese-text-segmentation
• machine-learning

Self-contained Japanese Morphological Analyzer written in pure Go

• japanese
• tokenizer
• nlp-library
• japanese-language

Pragmatic Segmenter is a rule-based sentence boundary detection gem that works out-of-the-box across many languages.

No-Frills Functional Programming Lib Augmenting Javascript/Node.js

• functional-programming
• pure-function
• recursion
• javascript

A multilingual tokenizer to split a string into tokens

Ruby port of the NLTK Punkt sentence segmentation algorithm

• rubynlp
• sentence-tokenizer
• ruby
• sentence-boundaries

A Go library for performing Unicode Text Segmentation as described in Unicode Standard Annex #29

A fast and accurate rule-based sentence segmentation tool for Ruby.