Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
25–48 of 298 · Most stars
YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
☁️ Build multimodal AI applications with cloud-native stack
- neural-search
- cloud-native
- deep-learning
- machine-learning
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
- pytorch
- speech-recognition
- paraformer
- punctuation
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Turns Data and AI algorithms into production-ready web applications in no time.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.
Workflow Engine for Kubernetes
- workflow
- kubernetes
- argo
- dag
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
- ai
- real-time
- voice
- voice-assistant
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
Access your entire server infrastructure from your local desktop
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
- workflow-schedule
- azkaban
- airflow
- task-scheduler
YouTube Music client for Android
- android
- innertube
- material-design
- material-ui
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
The official Open-Asset-Importer-Library Repository. Loads 40+ 3D-file-formats into one unified and clean data structure.
- asset-pipeline
- assets
- game-development
- c-plus-plus
Proxy based Redis cluster solution supporting pipeline and scaling dynamically
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
Streamlink is a CLI utility which pipes video streams from various services into a video player
- streamlink
- python
- livestream
- twitch
Kubescape is an open-source Kubernetes security platform for your IDE, CI/CD pipelines, and clusters. It includes risk analysis, security, compliance, and misconfiguration scanning, saving Kubernetes users and administrators precious time, effort, and resources.
- kubernetes
- security
- nsa
- mitre-attack
Connect APIs, remarkably fast. Free for developers.
- eventsourcing
- event-sourcing
- automation
- workflow
Spug is a lightweight agent-free automatic operation and maintenance platform designed for small and medium-sized enterprises. It integrates host management, host batch execution, host online terminal, file management, application release and deployment, pipelines, online task planning, configuration center, monitoring, alarm and so on.
- ops
- operations
- devops
- webssh