Repository Index

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

• ray
• distributed
• parallel
• machine-learning

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models

Migrate to PostgreSQL in a single command!

• postgresql
• migration
• mysql
• mssql

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

• data
• python
• data-engineering
• data-lake

Concurrent and multi-stage data ingestion and data processing with Elixir

• elixir
• genstage
• data-ingestion
• data-processing

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

Embulk: Pluggable Bulk Data Loader.

• embulk
• bulk-loader

Amazon Redshift Database Loader implemented in AWS Lambda

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

go-etl is a toolset for data extraction, transformation and loading.

• etl
• mysql
• go
• postgresql

Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.

Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk

• clickhouse
• data-engineering
• data-generation
• devops

EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.

• dotnet
• dotnet-core
• etl
• etl-framework

Gatling plugin for Apache Kafka — produce, request-reply, and consume load testing with Avro, Protobuf, and Schema Registry support

• gatling
• kafka
• performance
• performance-testing

Gatling plugin for AMQP/RabbitMQ — publish, request-reply, and consume patterns with connection pooling

• amqp
• gatling
• gatling-plugin
• performance

AWS_MSK_IAM Authentication Plugin for Broadway Kafka