Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
47,133 repositories indexed
1–22 of 22 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
🍴 Web applications made easy. Since 2011.
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.
- scala
- sqs-interface
- elasticmq
- amazon-sqs
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
Run MapReduce jobs on Hadoop or Amazon Web Services
Barebone MQTT broker that can run on any stream server, the node way
EventMesh is a new generation serverless event middleware for building distributed event-driven applications.
- esb
- event-driven
- event-mesh
- event-governance
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
- data-analytics
- data-cleaning
- data-cleansing
- data-engineering
A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
- data-ops
- data-quality
- data-quality-checks
- data-quality-measurement
Bots-as-a-Service for Slack
Database synchronizer for Magento 2 (and Wordpress), based on Magerun2. Keeping development, staging and production in sync easily.
A go library for easy configure and run command chains. Such like pipelining in unix shells.
- golang
- go
- library
- command
Scramjet Platform is a distributed runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters without exposing network ports to the public internet.
- nodejs
- stream-processing
- transform-as-a-service
- real-time
Lightweight DataFrame validation decorators for Pandas, Polars, Modin, and PyArrow. No custom types required.
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
- ruby
- rake
- pwrake
- parallel
Declarative data quality engine. Define checks in YAML, run anywhere.
- airflow
- anomaly-detection
- data-contracts
- data-quality
🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines
- etl
- etl-framework
- etl-pipeline
- python