🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- nlp
- natural-language-processing
- pytorch
- pytorch-transformers
Repository Index
47,133 repositories indexed
1–23 of 23 · Most stars
See, hear, and interact on your local devices.
- minicpm
- minicpm-v
- multi-modal
- minicpm-o
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
- nlp
- pre-trained-model
- unilm
- minilm
☁️ Build multimodal AI applications with cloud-native stack
- neural-search
- cloud-native
- deep-learning
- machine-learning
Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
- machine-translation
- speaker-recognition
- asr
- tts
Janus-Series: Unified Multimodal Understanding and Generation Models
- any-to-any
- foundation-models
- llm
- multimodal
🚀 Truly open-source AI avatar(digital human) toolkit for offline video generation and digital human cloning.
- ai-avatar
- ai-avatars
- digital-human
- video-generation
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
An open source implementation of CLIP.
- deep-learning
- pytorch
- computer-vision
- language-model
🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
- bert
- sentence-encoding
- deep-learning
- clip-model
Visualize, query, and stream to train on multimodal robotics data.
Production ready toolkit to run AI locally
Translated[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o
- image-classification
- image-text-retrieval
- llm
- semantic-segmentation
Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.
NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
A modular active learning framework for Python
- scikit-learn
- machine-learning
- python
- active-learning
A Multilingual, Multimodal Translation Model Family
- llm
- machine-translation
Fusing Histology and Genomics via Deep Learning - IEEE TMI
- pathomic
- fusion
- multimodal
- pathology
PyTorch implementation of the MIDAS algorithm for single-cell multimodal data integration (Nature Biotechnology 2024).
- deep-generative-model
- deep-learning
- single-cell-data-integration
- single-cell-multiomics
Deep learning model for single-cell inference of multi-omic profiles from a single input modality.
- single-cell
- scrna-seq
- scatac-seq
- deep-neural-networks
A Go toolkit for building AI agents and applications across multiple providers. Unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking.
- ai
- golang
- llm
- structured-output