Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
Topic Modelling for Humans
An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.
- python
- pii
- privacy
- data-anonymization
general natural language facilities for node
Leveraging BERT and c-TF-IDF to create easily interpretable topics.
An open source library for deep learning end-to-end dialog systems and chatbots.
An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more
Translated👮♂️The sensitive word tool for java.(Sensitive words / prohibited words / illegal words / profanity. A high-performance Java sensitive word filtering tool framework implemented based on the DFA algorithm. Built-in support for tag classification and grading of words. Please do not post content involving politics, advertising, marketing, circumventing the Great Firewall, or violating national laws and regulations. A high-performance sensitive word detection and filtering component, with traditional/simplified Chinese conversion, full-width/half-width conversion, Chinese character to pinyin conversion, fuzzy search, and other features.)
- sensitive
- sensitive-word
- sensitive-word-filter
- dfa
Local-first healthcare AI: clinical NER and HIPAA PII de-identification on hardware you control. 2,200+ medical models, 35 model-backed PII languages, and Python, MLX, Android and browser runtimes. Apache-2.0 SDK.
- healthcare
- llm
- on-device
- on-premise
TranslatedChinese text classification, TextCNN, TextRNN, FastText, TextRCNN, BiLSTM_Attention, DPCNN, Transformer, based on pytorch, ready to use out of the box.
TranslatedChinese characters to pinyin converter (pypinyin)
- python
- pinyin
- pypinyin
- hanzi
TranslatedA high-performance sensitive word (illegal word / profanity) detection and filtering component, which also comes with Traditional/Simplified Chinese conversion, and supports full-width/half-width conversion, Chinese character to pinyin conversion, fuzzy search, and other features.
- word-filter
- stringsearch
- wordssearch
- pinyin
Easily train your own text-generating neural network of any size and complexity on any text dataset with a few lines of code.
Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services
- bert
- ner
- named-entity-recognition
- blstm
🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines
- rag
- chonkie
- chunker
- chunking-algorithm
Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
- transformers
- text-classification
- named-entity-recognition
- question-answering
Snips Python library to extract meaning from text
Facilitating the design, comparison and sharing of deep text matching models.
SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm
- levenshtein
- fuzzy-search
- approximate-string-matching
- edit-distance
fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
Top2Vec learns jointly embedded topic, document and word vectors.
- topic-modeling
- word-embeddings
- document-embedding
- topic-vector
Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo
🇨🇳 功能全面的汉字工具库 (拼音 笔画 偏旁 成语 语音 可视化等) (Chinese character util)
- spell-stroke
- draw
- chinese-characters
- pinyin