Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
Logstash - transport and process your logs, events, or other data
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
This repository is a getting started guide to Singer.
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
The Frank!Framework is an easy-to-use, stateless integration framework which allows (transactional) messages to be modified and exchanged between different systems.
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
High performant ETL engine written in TypeScript
🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines