Dex : The Data Explorer -- A data visualization tool written in Java/Groovy/JavaFX capable of powerful ETL and publishing web visualizations.
Repository Index
77,852 visible repositories in the index
73–96 of 122 · Most stars
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
- transform
- etl
- xml
- json
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
- change-data-capture
- data
- data-collection
- data-engineering
The MongoDB Spark Connector
The simple, stupid batch framework for Java
- java
- batch-processing
- batch
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
- go
- data-integration
- etl
- data-pipeline
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
- porter
- data-import
- framework
- data-transformation
SmartCode = IDataSource -> IBuildTask -> IOutput => Build Everything!!!
- code-generator
- etl
- dotnet-core
- dotnet
🌱 The ultimate solution for populating your MongoDB database.
Distributed Data Transfer Service for MySQL
- mysql
- data-transfer
- data-migration
- replication
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
- java
- smooks
- enterprise-integration
- etl
The data platform for manufacturing
- oee
- iot
Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs
- spark
- neo4j-connector
- cypher
- neo4j-driver
The data transformation tool the open-source community deserves.
- data-engineering
- duckdb
- data-lineage
- data-pipeline
Powerful CSV & Excel Import experience for SaaS 🚀 Save months building data import experience from scratch 💰
- csv-import
- data-import
- data-mapping
- data-review
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
- graphrag
- information-extraction
- knowledge-graph
- llm
📜 A CLI toolkit for extracting and working with your digital history
- chronicle
- chronicle-etl
- personal-data
- etl
Blockchain Indexing Engine
go-etl is a toolset for data extraction, transformation and loading.
- etl
- mysql
- go
- postgresql
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
Event-driven OneLake pipeline with automated Upsert merge and Python reporting
- activator
- data-engineering
- data-factory
- data-pipeline