Ranked by Attention Score: stars gained over the last 90 complete days
48 repositories by Attention Score · Aug 5, 2026 – Nov 2, 2026 (UTC)
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
Apache Superset is a Data Visualization and Data Exploration Platform
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Learn how to develop, deploy and iterate on production-grade ML applications.
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Incremental engine for long horizon agents 🌟 Star if you like it!
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
An orchestration platform for the development, production, and observation of data assets.
Open Source Feature Flags, Experimentation, and Product Analytics
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
The Data Engineering Cookbook
Business intelligence as code: build fast, interactive data visualizations in SQL and markdown
Always know what to expect from your data.
Workflow Engine for Kubernetes
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
The Open Source Feature Store for AI/ML
Turns Data and AI algorithms into production-ready web applications in no time.
🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
lakeFS - Data version control for your data lake | Git for data
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Fancy stream processing made operationally mundane
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
Privacy and Security focused Segment-alternative, in Golang and React
Roadmap to becoming a data engineer in 2021
Python Stream Processing
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
Python Streaming DataFrames for Kafka
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
Compare tables within or across databases
Clean APIs for data cleaning. Python implementation of R package Janitor
Reverse ETL for the code-first data stack
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.
Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.
Data ingestion engine — Singer taps to Parquet in cloud buckets
Polyglot workflows without leaving the comfort of your technology stack.