🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
SGLang is a high-performance serving framework for large language models and multimodal models.
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
☁️ Build multimodal AI applications with cloud-native stack
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
Janus-Series: Unified Multimodal Understanding and Generation Models
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
A framework to enable a multimodal model to operate a computer.
FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
A Multilingual, Multimodal Translation Model Family
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
Official Code for OpenVLHarness: Real-World Visual Agentic Tool‑Use with Multimodal Memory
The Complete Street Rule for ArcGIS CityEngine is a scenario oriented design tool intended to enable users to quickly create procedurally generated multimodal streets.
PyTorch implementation of the MIDAS algorithm for single-cell multimodal data integration (Nature Biotechnology 2024).
Official implementation of JAMIE, Joint variational Autoencoders for Multimodal Imputation and Embedding
scPair: Boosting single cell multimodal analysis by leveraging single cell atlases and implicit feature selection