NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
Repository Index
47,133 repositories indexed
25–43 of 43 · Most stars
The Ruby AI framework.
A Multilingual, Multimodal Translation Model Family
- llm
- machine-translation
Recent Advances in Vision and Language PreTrained Models (VL-PTMs)
- vision-and-language
- pretraining
- multimodal-deep-learning
- bert
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
Papers, code and datasets about deep learning and multi-modal learning for video analysis
- deep-learning
- video-analysis
- paper
- multimodal-learning
Stop paying the multimodal tax. Your AI agent keeps every image it has seen but only sends the ones it needs: lighter chats, no endless reconnecting, no starting over. Works with Codex, Claude Code, Cline, Gemini CLI, OpenCode, WorkBuddy and more. Install once, forget it.
- codex
- codex-plugin
- context-management
- developer-tools
multimodal routing, geocoding, and map tiles
- routing
- routing-engine
- gtfs
- intermodal
Scene-aware storyboard for DeepSeek Harness: 9-16 numbered cells with timestamps + transcript + an audio envelope, packed into ONE image (384 tokens). Two tools: video_storyboard and video_zoom.
- asr
- contact-sheet
- deepseek-harness
- dsh-plugin
Reference mapping for single-cell genomics
- single-cell
- deep-learning
- rna-seq-analysis
- data-integration
Developed to power Conveyal's web-based interface for scenario planning and land-use/transport accessibility analysis, R5 is our routing engine for multimodal (transit/bike/walk/car) networks with a particular focus on public transit
Official Code for OpenVLHarness: Real-World Visual Agentic Tool‑Use with Multimodal Memory
- agent-harness
- agentic-ai
- ai-agents
- coding-agent
Fusing Histology and Genomics via Deep Learning - IEEE TMI
- pathomic
- fusion
- multimodal
- pathology
Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning - Cancer Cell
- histology
- wsi-images
- pan-cancer-analysis
- genomics
The Complete Street Rule for ArcGIS CityEngine is a scenario oriented design tool intended to enable users to quickly create procedurally generated multimodal streets.
- complete
- streets
- transportation
- cityengine
PyTorch implementation of the MIDAS algorithm for single-cell multimodal data integration (Nature Biotechnology 2024).
- deep-generative-model
- deep-learning
- single-cell-data-integration
- single-cell-multiomics
A Go toolkit for building AI agents and applications across multiple providers. Unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking.
- ai
- golang
- llm
- structured-output
Official implementation of JAMIE, Joint variational Autoencoders for Multimodal Imputation and Embedding
- autoencoder
- imputation
- integration
- multimodal
scPair: Boosting single cell multimodal analysis by leveraging single cell atlases and implicit feature selection