47,133 repositories indexed
1–24 of 34 · Most stars
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Topic Modelling for Humans
Pretrained models for TensorFlow.js
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
modest natural-language processing
Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Open-source offline translation library written in Python
Module for automatic summarization of text documents and HTML pages.
Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining
A full spaCy pipeline and models for scientific/biomedical documents.
Self-contained Machine Learning and Natural Language Processing library in Go
Translated"Jieba" (Chinese for "to stutter") Chinese text segmentation: built to be the best PHP Chinese word segmentation module.
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
Pretrained model hub for Keras 3.
Fast vectorization, topic modeling, distances and GloVe word embeddings in R.
🚀 Catalyst is a C# Natural Language Processing library built for speed. Inspired by spaCy's design, it brings pre-trained models, out-of-the box support for training word and document embeddings, and flexible entity recognition models.
Parse human names into title, given, middle, family, suffix, nickname and maiden name. Typed, zero-dependency, CJK-aware.
A Modern C++ Data Sciences Toolkit
Translated斑海豹 (Spotted Seal) — Phocinae-Largha-150M-v1: a 144.3M bilingual decision model. Tiny model. Big decisions.
CogComp's Natural Language Processing Libraries and Demos: Modules include lemmatizer, ner, pos, prep-srl, quantifier, question type, relation-extraction, similarity, temporal normalizer, tokenizer, transliteration, verb-sense, and more.