Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
110,088 visible repositories in the index
1–18 of 18 · Most stars
Apache Spark - A unified analytics engine for large-scale data processing
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
Apache Beam is a unified programming model for Batch and Streaming data processing.
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
Data-Centric Pipelines and Data Versioning
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Apache Ignite
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
The official repository for ROOT: analyzing, storing and visualizing big data, scientifically
A Scala API for Apache Beam and Google Cloud Dataflow.
YTsaurus is a scalable and fault-tolerant open-source big data platform.
Apache Hadoop docker image
Apache DataFusion Ballista Distributed Query Engine
Apache Spark docker image
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Baidu Bigflow is an interface that allows for writing distributed computing programs and provides lots of simple, flexible, powerful APIs. Using Bigflow, you can easily handle data of any scale. Bigflow processes 4P+ data inside Baidu and runs about 10k jobs every day.