🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- nlp
- natural-language-processing
- pytorch
- pytorch-transformers
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
☁️ Build multimodal AI applications with cloud-native stack
- neural-search
- cloud-native
- deep-learning
- machine-learning
Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
- machine-translation
- speaker-recognition
- asr
- tts
Janus-Series: Unified Multimodal Understanding and Generation Models
- any-to-any
- foundation-models
- llm
- multimodal
🚀 Truly open-source AI avatar(digital human) toolkit for offline video generation and digital human cloning.
- ai-avatar
- ai-avatars
- digital-human
- video-generation
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Visualize, query, and stream to train on multimodal robotics data.
Production ready toolkit to run AI locally
Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.
NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
A Multilingual, Multimodal Translation Model Family
- llm
- machine-translation
Papers, code and datasets about deep learning and multi-modal learning for video analysis
- deep-learning
- video-analysis
- paper
- multimodal-learning
Reference mapping for single-cell genomics
- single-cell
- deep-learning
- rna-seq-analysis
- data-integration
Fusing Histology and Genomics via Deep Learning - IEEE TMI
- pathomic
- fusion
- multimodal
- pathology
Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning - Cancer Cell
- histology
- wsi-images
- pan-cancer-analysis
- genomics
PyTorch implementation of the MIDAS algorithm for single-cell multimodal data integration (Nature Biotechnology 2024).
- deep-generative-model
- deep-learning
- single-cell-data-integration
- single-cell-multiomics
A Go toolkit for building AI agents and applications across multiple providers. Unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking.
- ai
- golang
- llm
- structured-output