47,133 repositories indexed
1–24 of 48 · Most stars
TranslatedChinese word segmentation, part-of-speech tagging, named entity recognition, dependency parsing, constituency parsing, semantic dependency parsing, semantic role labeling, coreference resolution, style transfer, semantic similarity, new word discovery, keyword and keyphrase extraction, automatic summarization, text classification and clustering, pinyin and Simplified/Traditional Chinese conversion, natural language processing
TranslatedJieba (结巴, "stutter") Chinese word segmentation
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
💫 Industrial-strength Natural Language Processing (NLP) in Python
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Offline, privacy-first grammar checker. Fast, open-source, Rust-powered
Topic Modelling for Humans
State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
Style and Grammar Checker for 25+ Languages
Pretrained models for TensorFlow.js
NLTK Source
A very simple framework for state-of-the-art Natural Language Processing (NLP)
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
modest natural-language processing
Unsupervised text tokenizer for Neural Network-based text generation.
Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
general natural language facilities for node
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages
Open-source offline translation library written in Python