An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
- workflow
- governance
- azkaban
- davinci
REST job server for Apache Spark
Confluent Schema Registry for Kafka
- schema-registry
- kafka
- schema
- schemas
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
Crossbar.io - WAMP application router
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Livy is an open source REST interface for interacting with Apache Spark from anywhere
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
Lua kafka client driver for the Openresty based on the cosocket API
Karapace - Your Apache Kafka® essentials in one tool
- kafka
- schema-registry
- rest-proxy
An open source, a high-performance hybrid integration platform that allows developers quick integration with any application, data, or system.
- integration
- middleware
- wso2
- esb
A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.
- isolation-forest
- spark
- outlier-detection
A library that brings useful functions from various modern database management systems to Apache Spark
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
- dotnet
- dotnet-core
- etl
- etl-framework
Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.
- postgres
- postgresql
- python
- infectious-disease
Quick integration toolkit for Ixian. Connects apps, IoT devices, AI Agents, and services to the Ixian Platform via REST APIs and message queues (MQTT/RabbitMQ). Provides developers with a simple bridge into secure Ixian communication and Client Discovery.
- api
- encryption
- integration
- iot