Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
47,133 repositories indexed
1–14 of 14 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
A unified ecosystem and workspace for large-scale federated learning, bridging the gap between centralized model evolution and decentralized edge intelligence.
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Distributed batch data processing framework
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
- ruby
- rake
- pwrake
- parallel
Common Lisp distributed computation framework implemented using Swank Client
The core libraries of the teknek stream processing platform