59,132 visible repositories in the index
25–48 of 52 · Most stars
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 426 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
Healthcare data infrastructure for next generation care delivery.
SmartCode = IDataSource -> IBuildTask -> IOutput => Build Everything!!!
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs
The data transformation tool the open-source community deserves.
Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.
📜 A CLI toolkit for extracting and working with your digital history
Blockchain Indexing Engine
A plugin to push to Algolia based on graphQl queries
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.
Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.
Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.
Flume NG MongoDB source.
Enables search with Algolia in Hugo static sites
Insert incoming MQTT values into InfluxDB. Follows mqtt-smarthome architecture.
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
Open source industrial IoT connectivity gateway.
Internet Archive's Sparkling Data Processing Library
R COBOL DI (Data Integration) Package : Import COBOL CopyBook data files directly into R as properly structured data frames.
Instant data for apps on Bitcoin.
Jmeter JTL parsing with Logstash for Elasticseacrh and Influxdb ...