Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Repository Index
47,133 repositories indexed
1–11 of 11 · Most stars
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
An orchestration platform for the development, production, and observation of data assets.
Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
Reverse ETL for the code-first data stack
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines
Data ingestion engine — Singer taps to Parquet in cloud buckets