Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
47,133 repositories indexed
1–24 of 26 · Most stars
Apache Flink
All-in-one IoT Platform - Device management, data collection, processing and visualization.
- visualization
- iot
- iot-platform
- mqtt
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
CMAK is a tool for managing Apache Kafka clusters
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
Distributed Big Data Orchestration Service
- big-data
- bigdata
- orchestration
- configuration
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
Sparkling Water provides H2O functionality inside Spark cluster
- h2o
- spark
- machine-learning
- integration
Mirror of Apache Samza
- scala
- big-data
- samza
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
- antelope
- api
- big-data
- blockchain
Big Data essentials: Hadoop, MapReduce, Spark. Explore tutorials and demos in Jupyter notebooks—most are self-contained and live, ready to run with a click.
A curated list of awesome Qlik extensions and resources for Qlik Sense and QlikView
- qlik
- qlik-sense
- qlik-extension
- qlik-sense-extension
A general-purpose data analysis engine radically changing the way batch and stream data is processed