Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
104,105 visible repositories in the index
1–18 of 18 · Most stars
Apache Spark - A unified analytics engine for large-scale data processing
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
Apache Beam is a unified programming model for Batch and Streaming data processing.
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
A lightweight data processing framework built on DuckDB and 3FS.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
A cluster computing framework for processing large-scale geospatial data
Python Stream Processing
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Distributed Stream Processing
Serverless batch computing made simple.
Data processing on Hadoop without the hassle.
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
Phoebus is a distributed framework for large scale graph processing written in Erlang.
Distributed batch data processing framework
The core libraries of the teknek stream processing platform