Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
142,273 visible repositories in the index
1–15 of 15 · Most stars
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Enterprise job scheduling middleware with distributed computing ability.
- scheduler
- workflow
- distributed
- mapreduce
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
A modular implementation of timely dataflow in Rust
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
A Scala API for Apache Beam and Google Cloud Dataflow.
- scala
- bigquery
- google-cloud
- beam
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Baidu Bigflow is an interface that allows for writing distributed computing programs and provides lots of simple, flexible, powerful APIs. Using Bigflow, you can easily handle data of any scale. Bigflow processes 4P+ data inside Baidu and runs about 10k jobs every day.
Python module that allows one to easily write and run Hadoop programs.
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.
Distributed batch data processing framework
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
- ruby
- rake
- pwrake
- parallel