Repository Index

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

• gpt-4
• chatbot
• chatgpt
• llama

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm

☁️ Build multimodal AI applications with cloud-native stack

• neural-search
• cloud-native
• deep-learning
• machine-learning

Janus-Series: Unified Multimodal Understanding and Generation Models

• any-to-any
• foundation-models
• llm
• multimodal

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

• llm
• lora
• sft
• multimodal

Visualize, query, and stream to train on multimodal robotics data.

• visualization
• computer-vision
• python
• robotics

Official implementation of JAMIE, Joint variational Autoencoders for Multimodal Imputation and Embedding

• autoencoder
• imputation
• integration
• multimodal