Repository Index

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

• nlp
• natural-language-processing
• pytorch
• pytorch-transformers

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

• fine-tuning
• llama
• llm
• peft

AiLearning：数据分析+机器学习实战+线性代数+PyTorch+NLTK+TF2

• fp-growth
• apriori
• mahchine-leaning
• naivebayes

微舆：人人可用的多Agent舆情分析助手，打破信息茧房，还原舆情原貌，预测未来走向，辅助决策！从0实现，不依赖任何框架。

• agent-framework
• data-analysis
• multi-agent-system
• nlp

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

• llm
• nlp
• python
• gemini-ai

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

• nlp
• natural-language-processing
• hanlp
• pos-tagging

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

• calibration
• classification
• decision-model
• huggingface

💫 Industrial-strength Natural Language Processing (NLP) in Python

• natural-language-processing
• data-science
• machine-learning
• python

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

• large-language-models
• nlp
• knowledge-curation
• naacl

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

• nlp
• datasets
• pytorch
• tensorflow

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

• nlp
• machine-learning
• machine-learning-library
• bot

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

• llm
• plm
• pre-trained-language-models
• alpaca

Topic Modelling for Humans

• gensim
• topic-modeling
• information-retrieval
• machine-learning

Nano vLLM

• inference
• llm
• pytorch
• transformer

NLTK Source

• nltk
• python
• nlp
• natural-language-processing

A very simple framework for state-of-the-art Natural Language Processing (NLP)

• pytorch
• nlp
• named-entity-recognition
• sequence-labeling

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

• python
• search
• nlp
• semantic-search

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

• nlp
• embedding
• bert
• ernie

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

• chinese-bert
• tensorflow
• pytorch
• bert

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

• python
• nlp
• natural-language-processing
• machine-learning

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

• web-scraping
• text-extraction
• nlp
• html2text

Open-source offline translation library written in Python

• python
• machine-translation
• transformers
• translation