Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
99,825 visible repositories in the index
1–11 of 11 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
Open-source software for volunteer computing and grid computing.
- boinc
- distributed-computing
- volunteer-computing
- high-performance-computing
YTsaurus is a scalable and fault-tolerant open-source big data platform.
- big-data
- clickhouse
- distributed-database
- lakehouse
Distributed Stream Processing
- stream-processing
- stream-processor
- stream-processing-engine
- pony-language
Lightning-fast cluster computing in Java, Scala and Python.
AI based entity resolution, identity resolution, master data management and deduplication at scale for humans and agents
- fuzzymatch
- fuzzy-matching
- deduplication
- dedupe
Cloud-native genomic dataframes and batch computing
- genetics
- vcf
- genomics
- gwas
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake