A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
Repository Index
132,183 visible repositories in the index
1–24 of 83 · Most stars
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
modest natural-language processing
- nlp
- part-of-speech
- named-entity-recognition
An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.
- python
- pii
- privacy
- data-anonymization
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
- nlp
- natural-language-processing
- natural-language-understanding
- language-model
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
- python
- machine-learning
- natural-language-processing
- web-mining
Transformer related optimization, including BERT, GPT
- pytorch
- transformer
- gpt
- bert
Local-first healthcare AI: clinical NER and HIPAA PII de-identification on hardware you control. 2,200+ medical models, 35 model-backed PII languages, and Python, MLX, Android and browser runtimes. Apache-2.0 SDK.
- healthcare
- llm
- on-device
- on-premise
Easily train your own text-generating neural network of any size and complexity on any text dataset with a few lines of code.
- keras
- deep-learning
- text-generation
- tensorflow
🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines
- rag
- chonkie
- chunker
- chunking-algorithm
Minimal keyword extraction with BERT
- keyword-extraction
- keyphrase-extraction
- bert
- mmr
Module for automatic summarization of text documents and HTML pages.
- python
- lsa
- textteaser
- html-page
📐 Compute distance between sequences. 30+ algorithms, pure python implementation, common interface, optional external libs usage.
- distance
- algorithm
- python
- textdistance
SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm
- levenshtein
- fuzzy-search
- approximate-string-matching
- edit-distance
Top2Vec learns jointly embedded topic, document and word vectors.
- topic-modeling
- word-embeddings
- document-embedding
- topic-vector
MITIE: library and tools for information extraction
- machine-learning
- natural-language-processing
- information-extraction
- python
Text preprocessing, representation and visualization from zero to hero.
- text-preprocessing
- text-representation
- text-visualization
- nlp
The most popular spellchecking library.
- natural-language-processing
- spellcheck
- spellchecker
- stemming
Beautiful visualizations of how language differs among document types.
- nlp
- d3
- word-embeddings
- machine-learning
A very simple BiLSTM-CRF model for Chinese Named Entity Recognition 中文命名实体识别 (TensorFlow)
- named-entity-recognition
- bilstm-crf-model
- tensorflow
Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining
ALIbaba's Collection of Encoder-decoders from MinD (Machine IntelligeNce of Damo) Lab