Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

• data-integration
• high-performance
• offline
• real-time

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

• python
• aws
• pandas
• apache-arrow

Self-hosted platform to unify wearable health data through one AI-ready API.

• wearables
• apple-health
• apple-watch
• fitness

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

• data
• ingestion
• replication
• management

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

• transform
• etl
• xml
• json

The simple, stupid batch framework for Java

• java
• batch-processing
• batch

SmartCode = IDataSource -> IBuildTask -> IOutput => Build Everything!!!

• code-generator
• etl
• dotnet-core
• dotnet

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

Powerful CSV & Excel Import experience for SaaS 🚀 Save months building data import experience from scratch 💰

• csv-import
• data-import
• data-mapping
• data-review

Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.

A high performance data transfer tool that can be used to transfer data from Bitcoin Core blockchain files to a SQL Server database

• bitcoin

Open source industrial IoT connectivity gateway.

• iotsharp
• iot
• modbus
• iotgateway

Java importer skeleton for complicated, business-logic-heavy high-performance Neo4j imports directly from SQL databases, CSV files, etc.

• neo4j
• java
• neo4j-graphaware-framework

Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.

• postgres
• postgresql
• python
• infectious-disease

Instant data for apps on Bitcoin.

• clarity
• stacks
• typescript
• blockchain