TranslatedDataX is the open-source version of Alibaba Cloud DataWorks Data Integration.
Repository Index
47,133 repositories indexed
1–22 of 22 · Most stars
Logstash - transport and process your logs, events, or other data
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
Pentaho Data Integration ( ETL ) a.k.a Kettle
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices
Database Subsetting and Relational Data Browsing Tool.
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
- data
- ingestion
- replication
- management
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Embulk: Pluggable Bulk Data Loader.
- embulk
- bulk-loader
Apache InLong - a one-stop, full-scenario integration framework for massive data
- inlong
- one-stop-service
- data-streaming
- event-streaming
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
The MongoDB Spark Connector
The simple, stupid batch framework for Java
- java
- batch-processing
- batch
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
- java
- smooks
- enterprise-integration
- etl
RDF-Centric Map/Reduce Framework and Freebase data conversion tool
Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.
Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.
Flume NG MongoDB source.
- flume
- sink
- mongodb
Massively parallel RDBMS to Neo4j Importer
Java importer skeleton for complicated, business-logic-heavy high-performance Neo4j imports directly from SQL databases, CSV files, etc.
- neo4j
- java
- neo4j-graphaware-framework
HBase adapters for Cascading
Read JMeter results in a csv file and put results in Influxdb database and allows you to create beautiful graphs with Grafana