Repository Index

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

• airflow
• apache
• apache-airflow
• python

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering

Turns Data and AI algorithms into production-ready web applications in no time.

• automation
• data-engineering
• data-ops
• data-visualization

An orchestration platform for the development, production, and observation of data assets.

• data-pipelines
• dagster
• workflow
• data-science

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

• python
• aws
• pandas
• apache-arrow

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

• etl
• data-integration
• python
• postgresql