Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
89,572 visible repositories in the index
1–18 of 18 · Most stars
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
Topic Modelling for Humans
An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.
Local-first healthcare AI: clinical NER and HIPAA PII de-identification on hardware you control. 2,200+ medical models, 35 model-backed PII languages, and Python, MLX, Android and browser runtimes. Apache-2.0 SDK.
Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services
Language, engine, and tooling for expressing, testing, and evaluating composable language rules on input strings.
⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.
This tool provides an efficient implementation of the continuous bag-of-words and skip-gram architectures for computing vector representations of words. These representations can be subsequently used in many natural language processing applications and for further research.
jiant is an nlp toolkit
Pragmatic Segmenter is a rule-based sentence boundary detection gem that works out-of-the-box across many languages.
Modular, fast NLP framework, compatible with Pytorch and spaCy, offering tailored support for French clinical notes.
JevStyle: local decision models for AI agents. Classify, route and score with a probability per option. systemone-compatible local server, MCP tools, Claude Code guard. Open weights, Apache-2.0.
An NLP library for Uralic languages such as Finnish, Skolt Sami, Moksha and so on. Also supporting some non-Uralic languages such as Spanish, French, Arabic, Swedish, Norwegian, Russian and English. LLMs, FSTs and More!
A multilingual tokenizer to split a string into tokens
A simple tokenizer in Ruby for NLP tasks.
Golang implementation of the Paice/Husk Stemming Algorithm
Surface realiser (part of a Natural Language Generation system) using Tree Adjoining Grammar