1.modelscope / ms-swiftUse PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).llmlorasftmultimodalpeftliger+13 morePythonAPACHE-2.016K total1.8K forks+243 starsfrom 15,765 · score 5.50
2.haotian-liu / LLaVA[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.gpt-4chatbotchatgptllamamultimodalllava+7 morePythonAPACHE-2.025K total2.8K forks+52 starsfrom 25,028 · score 3.97
3.deepseek-ai / JanusJanus-Series: Unified Multimodal Understanding and Generation Modelsany-to-anyfoundation-modelsllmmultimodalvision-language-pretrainingunified-modelPythonMIT18K total2.2K forks+31 starsfrom 17,735 · score 3.47
4.microsoft / unilmLarge-scale Self-supervised Pre-training Across Tasks, Languages, and Modalitiesnlppre-trained-modelunilmminilmlayoutlmlayoutxlm+14 morePythonMIT22K total2.7K forks+24 starsfrom 22,214 · score 3.22
5.jina-ai / serve☁️ Build multimodal AI applications with cloud-native stackneural-searchcloud-nativedeep-learningmachine-learningframeworkgrpc+14 morePythonAPACHE-2.022K total2.2K forks+10 starsfrom 21,840 · score 2.40