🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Repository Index
88,573 visible repositories in the index
1–15 of 15 · Most stars
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
Open-source software for volunteer computing and grid computing.
- boinc
- distributed-computing
- volunteer-computing
- high-performance-computing
YTsaurus is a scalable and fault-tolerant open-source big data platform.
- big-data
- clickhouse
- distributed-database
- lakehouse
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
- spark
- dask
- data-practitioners
- machine-learning
Python 3.8+ toolbox for submitting jobs to Slurm
- slurm
- python
- clusters
Distributed Stream Processing
- stream-processing
- stream-processor
- stream-processing-engine
- pony-language
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Distributed batch data processing framework
Tuple MapReduce for Hadoop: Hadoop API made easy
Web application for distributed compute analysis of Archive-It web archive collections.
The core libraries of the teknek stream processing platform