47,133 repositories indexed
1–24 of 35 · Most stars
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Privacy and Security focused Segment-alternative, in Golang and React
LFX Community Data Platform (CDP)
Self-hosted platform to unify wearable health data through one AI-ready API.
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
visual data prep powered by python
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
The data platform for manufacturing
The data transformation tool the open-source community deserves.
High performance Kafka consumer for InfluxDB. Supports collectd message formats.