🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- nlp
- natural-language-processing
- pytorch
- pytorch-transformers
111,820 visible repositories in the index
1–24 of 203 · Most stars
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Topic Modelling for Humans
State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
Style and Grammar Checker for 25+ Languages
Pretrained models for TensorFlow.js
A very simple framework for state-of-the-art Natural Language Processing (NLP)
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
Unsupervised text tokenizer for Neural Network-based text generation.
Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
general natural language facilities for node
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
Library for conversion between Traditional and Simplified Chinese
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Leveraging BERT and c-TF-IDF to create easily interpretable topics.
Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages
An open source library for deep learning end-to-end dialog systems and chatbots.
TranslatedThe pkuseg toolkit for multi-domain Chinese word segmentation
Open-source offline translation library written in Python