Repository Index

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

• ray
• distributed
• parallel
• machine-learning

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models

Migrate to PostgreSQL in a single command!

• postgresql
• migration
• mysql
• mssql

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

• data
• python
• data-engineering
• data-lake

Concurrent and multi-stage data ingestion and data processing with Elixir

• elixir
• genstage
• data-ingestion
• data-processing

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

Embulk: Pluggable Bulk Data Loader.

• embulk
• bulk-loader

Amazon Redshift Database Loader implemented in AWS Lambda

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

k6 extension to load test Apache Kafka with support for various serialization formats, SASL, TLS, compression, Schema Registry client and beyond

• k6
• kafka
• apache-kafka
• load-testing

go-etl is a toolset for data extraction, transformation and loading.

• etl
• mysql
• go
• postgresql

For SwiftUI apps - a primitive component to make Chat-UI with stable prepending - no scroll jumps when loading older messages.

• chat
• messaging
• swiftui

Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk

• clickhouse
• data-engineering
• data-generation
• devops

EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.

• dotnet
• dotnet-core
• etl
• etl-framework

Gatling plugin for Apache Kafka — produce, request-reply, and consume load testing with Avro, Protobuf, and Schema Registry support

• gatling
• kafka
• performance
• performance-testing

Gatling plugin for AMQP/RabbitMQ — publish, request-reply, and consume patterns with connection pooling

• amqp
• gatling
• gatling-plugin
• performance

AWS_MSK_IAM Authentication Plugin for Broadway Kafka