Repository Index

Self-contained Japanese Morphological Analyzer written in pure Go

• japanese
• tokenizer
• nlp-library
• japanese-language

A multilingual command line sentence tokenizer in Golang

• sentence-tokenizer
• tokenizer
• cli
• sentences

Bayesian text classifier with flexible tokenizers and storage backends for Go

Tokenizer (lexer) for anything

• golang
• tokenizer
• parse
• tokenizing

A tokenizer based on the dictionary and Bigram language models for Go. (Now only support chinese segmentation)

• tokenizer
• segmentation
• golang

The shamoji (杓文字) is a word filtering package

• go
• tokenizer
• word-filter