Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
47,133 repositories indexed
1–24 of 33 · Most stars
Apache Flink
All-in-one IoT Platform - Device management, data collection, processing and visualization.
- visualization
- iot
- iot-platform
- mqtt
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
CMAK is a tool for managing Apache Kafka clusters
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
Distributed Big Data Orchestration Service
- big-data
- bigdata
- orchestration
- configuration
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
Sparkling Water provides H2O functionality inside Spark cluster
- h2o
- spark
- machine-learning
- integration
Mirror of Apache Samza
- scala
- big-data
- samza
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
High-performance, time-indexed object storage for robotics and industrial IoT
- blob-storage
- storage-engine
- storage
- historian
Serverless proxy for Spark cluster
Tree diagrams with JavaScript :evergreen_tree: :chart_with_upwards_trend:
- tree
- javascript
- visualization
- big-data
Read and write Neuroglancer datasets programmatically.
- neuroglancer
- volumetric-data
- python
- cloud
BENERATOR is a leading software solution to generate, obfuscate, pseudonymize and migrate data for development, testing, and training purposes with a model-driven approach.
- testdata
- anonymization
- migration
- performance-testing