🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- nlp
- natural-language-processing
- pytorch
- pytorch-transformers
Repository Index
47,133 repositories indexed
1–24 of 101 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
🧠 Train a 64M-parameter LLM from scratch in just 2h!
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
A generative speech model for daily dialogue.
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more
- abliteration
- llm
- transformer
- censorship-resistance
SGLang is a high-performance serving framework for large language models and multimodal models.
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理
- nlp
- natural-language-processing
- hanlp
- pos-tagging
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
- calibration
- classification
- decision-model
- huggingface
💫 Industrial-strength Natural Language Processing (NLP) in Python
An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
- large-language-models
- nlp
- knowledge-curation
- naacl
A powerful MCP toolkit for coding, providing semantic retrieval and editing capabilities - the IDE for your agent
Code and documentation to train Stanford's Alpaca models, and generate the data.
- deep-learning
- instruction-following
- language-model
Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.
The #1 AI Harness for Building Resumes, PDFs, Cover Letters & more, locally with 100+ LLMs support.
- text-similarity
- natural-language-processing
- word-embeddings
- python
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
- semantic-search
- information-retrieval
- ai
- python
Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.
- audio-generation
- gpt-4o
- text-to-speech
- tts
Universal LLM Deployment Engine with ML Compilation
- llm
- machine-learning-compilation
- language-model
- tvm
Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.
- natural-language-processing
- machine-learning
- named-entity-recognition
- machine-translation
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm