Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
47,133 repositories indexed
1–12 of 12 · Most stars
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
Parallel computing with task scheduling
- dask
- python
- pydata
- numpy
Lightweight and extensible compatibility layer between dataframe libraries!
Distributed Deep learning with Keras & Spark
PySpark + Scikit-learn = Sparkit-learn
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!
The world's cleanest AutoML library ✨ - Do hyperparameter tuning with the right pipeline abstractions to write clean deep learning production pipelines. Let your pipeline steps have hyperparameter spaces. Design steps in your pipeline like components. Compatible with Scikit-Learn, TensorFlow, and most other libraries, frameworks and MLOps environments.
- pipeline
- pipeline-framework
- machine-learning
- deep-learning
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake
AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.
- aws-cli
- athena-cli
- aws-athena
- python
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.