47,133 repositories indexed
1–24 of 160 · Most stars
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
IM Chat OpenClaw
Apache Hadoop
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
Open-Source Web UI for Apache Kafka Management
Always know what to expect from your data.
🥊 Tiny 200 byte functional event emitter / pubsub.
Incremental engine for long horizon agents 🌟 Star if you like it!
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Apache Iceberg
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
Distributed Application Framework for .NET