Repository Index

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

• nlp
• natural-language-processing
• pytorch
• pytorch-transformers

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

• gpt-4
• chatbot
• chatgpt
• llama

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm

☁️ Build multimodal AI applications with cloud-native stack

• neural-search
• cloud-native
• deep-learning
• machine-learning

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

• machine-translation
• speaker-recognition
• asr
• tts

Janus-Series: Unified Multimodal Understanding and Generation Models

• any-to-any
• foundation-models
• llm
• multimodal

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

• llm
• lora
• sft
• multimodal

NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.

A Multilingual, Multimodal Translation Model Family

• llm
• machine-translation

PyTorch implementation of the MIDAS algorithm for single-cell multimodal data integration (Nature Biotechnology 2024).

• deep-generative-model
• deep-learning
• single-cell-data-integration
• single-cell-multiomics