Self-contained Japanese Morphological Analyzer written in pure Go
- japanese
- tokenizer
- nlp-library
- japanese-language
Repository Index
47,133 repositories indexed
1–6 of 6 · Most stars
A multilingual command line sentence tokenizer in Golang
- sentence-tokenizer
- tokenizer
- cli
- sentences
Bayesian text classifier with flexible tokenizers and storage backends for Go
Tokenizer (lexer) for anything
- golang
- tokenizer
- parse
- tokenizing
A tokenizer based on the dictionary and Bigram language models for Go. (Now only support chinese segmentation)
- tokenizer
- segmentation
- golang
The shamoji (杓文字) is a word filtering package
- go
- tokenizer
- word-filter