A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
Repository Index
142,273 visible repositories in the index
1–24 of 229 · Most stars
TranslatedChinese word segmentation, part-of-speech tagging, named entity recognition, dependency parsing, constituency parsing, semantic dependency parsing, semantic role labeling, coreference resolution, style transfer, semantic similarity, new word discovery, keyword and keyphrase extraction, automatic summarization, text classification and clustering, pinyin and Simplified/Traditional Chinese conversion, natural language processing
- nlp
- natural-language-processing
- hanlp
- pos-tagging
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
- calibration
- classification
- decision-model
- huggingface
💫 Industrial-strength Natural Language Processing (NLP) in Python
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
Unsupervised text tokenizer for Neural Network-based text generation.
- neural-machine-translation
- natural-language-processing
- word-segmentation
An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.
- python
- pii
- privacy
- data-anonymization
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
- natural-language-processing
- nlp
- nlp-parsing
- named-entity-recognition
Library for conversion between Traditional and Simplified Chinese
- chinese-translation
- traditional-chinese
- simplified-chinese
- chinese
Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.
- nlp
- nltk
- pattern
- python
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
- python
- machine-learning
- natural-language-processing
- web-mining
all kinds of text classification models and more with deep learning
- classification
- nlp
- fasttext
- textcnn
Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
- deep-learning
- pytorch
- machine-translation
- neural-machine-translation
An open source library for deep learning end-to-end dialog systems and chatbots.
Officially maintained, supported by PaddlePaddle, including CV, NLP, Speech, Rec, TS, big models and so on.
- paddlepaddle
- deep-learning
- neural-network
- computer-vision
TranslatedThe pkuseg toolkit for multi-domain Chinese word segmentation
- chinese-word-segmentation
Open-source offline translation library written in Python
- python
- machine-translation
- transformers
- translation
An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more
Transformer related optimization, including BERT, GPT
- pytorch
- transformer
- gpt
- bert
Local-first healthcare AI: clinical NER and HIPAA PII de-identification on hardware you control. 2,200+ medical models, 35 model-backed PII languages, and Python, MLX, Android and browser runtimes. Apache-2.0 SDK.
- healthcare
- llm
- on-device
- on-premise