🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- nlp
- natural-language-processing
- pytorch
- pytorch-transformers
104,105 visible repositories in the index
1–24 of 71 · Most stars
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
A very simple framework for state-of-the-art Natural Language Processing (NLP)
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
TranslatedThe pkuseg toolkit for multi-domain Chinese word segmentation
Translated👮♂️The sensitive word tool for java.(Sensitive words / prohibited words / illegal words / profanity. A high-performance Java sensitive word filtering tool framework implemented based on the DFA algorithm. Built-in support for tag classification and grading of words. Please do not post content involving politics, advertising, marketing, circumventing the Great Firewall, or violating national laws and regulations. A high-performance sensitive word detection and filtering component, with traditional/simplified Chinese conversion, full-width/half-width conversion, Chinese character to pinyin conversion, fuzzy search, and other features.)
VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines
Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research
State of the Art Natural Language Processing
Facilitating the design, comparison and sharing of deep text matching models.
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
🇨🇳 功能全面的汉字工具库 (拼音 笔画 偏旁 成语 语音 可视化等) (Chinese character util)
Text preprocessing, representation and visualization from zero to hero.
Go efficient fast multilingual NLP and text segmentation; support English, Chinese, Japanese and others.
Neologism dictionary based on the language resources on the Web for mecab-ipadic
The most popular spellchecking library.
natural language processor powered by plugins part of the @unifiedjs collective
⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/
EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit