Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Repository Index
108,094 visible repositories in the index
1–7 of 7 · Most stars
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Apache DataFusion Comet Spark Accelerator
- arrow
- datafusion
- rust
- spark
Baidu Bigflow is an interface that allows for writing distributed computing programs and provides lots of simple, flexible, powerful APIs. Using Bigflow, you can easily handle data of any scale. Bigflow processes 4P+ data inside Baidu and runs about 10k jobs every day.
Distribute work across machines using the lparallel API.
Common Lisp distributed computation framework implemented using Swank Client