dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
- dbt-viewpoint
- slack
- pypa
- data-modeling
132,241 visible repositories in the index
1–24 of 35 · Most stars
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Privacy and Security focused Segment-alternative, in Golang and React
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
Database Subsetting and Relational Data Browsing Tool.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
MongoShake is a universal data replication platform based on MongoDB's oplog. Redundant replication and active-active replication are two most important functions. 基于mongodb oplog的集群复制工具,可以满足迁移和同步的需求,进一步实现灾备和多活功能。
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
Extract Transform Load for Python 3.5+
cryo is the easiest way to extract blockchain data to parquet, csv, json, or python dataframes
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Actively maintained successor to Alibaba DataX — a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.
Python Extract Transform and Load Tables of Data
Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
A collection of Kafka Connect sinks and sources maintained by Lenses.io.
The software used to extract structured data from Wikipedia
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
Distributed Data Transfer Service for MySQL