1.modelscope / ms-swiftUse PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).llmlorasftmultimodalpeftliger+13 morePythonAPACHE-2.016K total1.8K forks+2,109 starsfrom 13,899 · score 7.65
2.haotian-liu / LLaVA[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.gpt-4chatbotchatgptllamamultimodalllava+7 morePythonAPACHE-2.025K total2.8K forks+495 starsfrom 24,585 · score 6.21
3.microsoft / unilmLarge-scale Self-supervised Pre-training Across Tasks, Languages, and Modalitiesnlppre-trained-modelunilmminilmlayoutlmlayoutxlm+14 morePythonMIT22K total2.7K forks+226 starsfrom 22,012 · score 5.42
4.deepseek-ai / JanusJanus-Series: Unified Multimodal Understanding and Generation Modelsany-to-anyfoundation-modelsllmmultimodalvision-language-pretrainingunified-modelPythonMIT18K total2.2K forks+215 starsfrom 17,551 · score 5.38
5.jina-ai / serve☁️ Build multimodal AI applications with cloud-native stackneural-searchcloud-nativedeep-learningmachine-learningframeworkgrpc+14 morePythonAPACHE-2.022K total2.2K forks+118 starsfrom 21,732 · score 4.78