Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
63,269 visible repositories in the index
1–24 of 26 · Most stars
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
Enterprise job scheduling middleware with distributed computing ability.
- scheduler
- workflow
- distributed
- mapreduce
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
Data-Centric Pipelines and Data Versioning
A unified ecosystem and workspace for large-scale federated learning, bridging the gap between centralized model evolution and decentralized edge intelligence.
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
A distributed task scheduler for Dask
- pydata
- dask
- distributed-computing
- python
Distributed Stream Processing
- stream-processing
- stream-processor
- stream-processing-engine
- pony-language
Phoebus is a distributed framework for large scale graph processing written in Erlang.
Distributed Temporal Graph Analytics with Apache Flink
- graph
- property-graph
- distributed-graph-analytics
- apache-flink
The repository host gives no description.
Standard API for Distributed Data Structures in R
Serverless ML Framework
Distributed batch data processing framework
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
- ruby
- rake
- pwrake
- parallel