The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
- agent
- vlm
- vision
- computer-use
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
- semantic-search
- information-retrieval
- ai
- python
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
- ai
- real-time
- voice
- voice-assistant
A framework to enable a multimodal model to operate a computer.
- automation
- openai
- pyautogui
FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
The Ruby AI framework.
Stop paying the multimodal tax. Your AI agent keeps every image it has seen but only sends the ones it needs: lighter chats, no endless reconnecting, no starting over. Works with Codex, Claude Code, Cline, Gemini CLI, OpenCode, WorkBuddy and more. Install once, forget it.
- codex
- codex-plugin
- context-management
- developer-tools
Scene-aware storyboard for DeepSeek Harness: 9-16 numbered cells with timestamps + transcript + an audio envelope, packed into ONE image (384 tokens). Two tools: video_storyboard and video_zoom.
- asr
- contact-sheet
- deepseek-harness
- dsh-plugin
Official Code for OpenVLHarness: Real-World Visual Agentic Tool‑Use with Multimodal Memory
- agent-harness
- agentic-ai
- ai-agents
- coding-agent
A Go toolkit for building AI agents and applications across multiple providers. Unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking.
- ai
- golang
- llm
- structured-output