[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
Repository Index
47,133 repositories indexed
1–14 of 14 · Most stars
:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
- instruction-tuning
- instruction-following
- large-vision-language-model
- visual-instruction-tuning
Janus-Series: Unified Multimodal Understanding and Generation Models
- any-to-any
- foundation-models
- llm
- multimodal
[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"
- object-detection
- open-world
- open-world-detection
- vision-language
Translated[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. An open-source multimodal dialogue model with performance close to GPT-4o
- image-classification
- image-text-retrieval
- llm
- semantic-segmentation
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
- privacy-first
- gguf
- llama-cpp
- local-ai
Recent Advances in Vision and Language PreTrained Models (VL-PTMs)
- vision-and-language
- pretraining
- multimodal-deep-learning
- bert
Stop paying the multimodal tax. Your AI agent keeps every image it has seen but only sends the ones it needs: lighter chats, no endless reconnecting, no starting over. Works with Codex, Claude Code, Cline, Gemini CLI, OpenCode, WorkBuddy and more. Install once, forget it.
- codex
- codex-plugin
- context-management
- developer-tools
Vision-Language Pathology Foundation Model - Nature Medicine
- bioimage-analysis
- bioimage-informatics
- conch
- foundation-model
Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
- agent-evaluation
- agentic-ai
- ai-agents
- awesome
A 4B grounding foundation model for physical intelligence. Unifies detection, referring, pointing, OCR, GUI, and layout grounding, achieving SOTA overall across 34 benchmarks and improving robot generalization and data efficiency in controlled evaluations.
- embodied-ai
- gui-grounding
- object-detection
- ocr
Companion repository for “In-Context Learning for Robots: Methods and Applications” — methods, datasets, and benchmarks organized by taxonomy and release date.
- awesome-list
- embodied-ai
- imitation-learning
- in-context-learning
Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System
- adversarial-attacks
- ai-safety
- autonomous-driving
- backdoor-attacks
Official code for "Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models" (ICRA 2026): Flow Policy Optimization for pi0
- aloha
- flow-matching
- icra2026
- jax