Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
79,841 visible repositories in the index
1–8 of 8 · Most stars
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
- asyncapi
- reactive
- specification
- websockets
Utils for streaming large files (S3, HDFS, gzip, bz2...)
- python
- s3
- hdfs
- webhdfs
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
- dataprep
- data-science
- datapreparation
- dataconnector
Messaging APIs for multi-platform
- messaging
- messenger
- line
- api-client
Clean APIs for data cleaning. Python implementation of R package Janitor
- pandas
- dataframe
- data
- cleaning-data
Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs
- spark
- neo4j-connector
- cypher
- neo4j-driver