Repository Index

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

• nlp
• natural-language-processing
• pytorch
• pytorch-transformers

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

• python
• pii
• privacy
• data-anonymization

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

• nlp
• nltk
• pattern
• python

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more

• nlp
• nlu
• nodejs
• sentiment-analysis

A machine learning software for extracting information from scholarly documents

• machine-learning
• scientific-articles
• pdf
• metadata

🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines

• rag
• chonkie
• chunker
• chunking-algorithm

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

• bert
• pre-training
• fine-tuning
• gpt

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

• deep-learning
• nlp
• transformer
• bert

Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining

敏感词过滤的几种实现+某1w词敏感词库

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

• nlp
• visualization
• explorables
• natural-language-processing

Microsoft.Recognizers.Text provides recognition and resolution of numbers, units, date/time, etc. in multiple languages (ZH, EN, FR, ES, PT, DE, IT, TR, HI, NL. Partial support for JA, KO, AR, SV). Packages available at: https://www.nuget.org/profiles/Recognizers.Text, https://www.npmjs.com/~recognizers.text

• nlp
• datetime
• timex
• parser-library

Self-contained Japanese Morphological Analyzer written in pure Go

• japanese
• tokenizer
• nlp-library
• japanese-language

No-Frills Functional Programming Lib Augmenting Javascript/Node.js

• functional-programming
• pure-function
• recursion
• javascript

JevStyle: local decision models for AI agents. Classify, route and score with a probability per option. systemone-compatible local server, MCP tools, Claude Code guard. Open weights, Apache-2.0.

• agent-skills
• calibration
• claude-code
• decision-model

Machine learning and natural language processing with Apache Pig

Multilingual library to easily parse date strings to java.util.Date objects.

• clojure
• parsing
• date-time
• date-parser

Combinatory Categorial Grammar (CCG): Full set CCG, and probabilistic CCG