Repository Index

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

• ray
• distributed
• parallel
• machine-learning

GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more

• go
• golang
• data-structure
• map

An orchestration platform for the development, production, and observation of data assets.

• data-pipelines
• dagster
• workflow
• data-science

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

• weak-supervision
• data-cleaning
• data-quality
• noisy-labels

Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.

• websocket
• redis
• real-time
• messaging

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

• aws
• gcp
• azure
• sql

a powerful mysql toolset with Go

A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others

• java
• bitset
• roaring-bitmaps
• roaringbitmap

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

• workflow
• governance
• azkaban
• davinci

Database Subsetting and Relational Data Browsing Tool.

• database
• subsetting
• frontend
• export

A complete, fully tested and documented data structure library written in pure JavaScript.

• javascript
• data-structures
• collections
• collection

Data structure and algorithm library for go, designed to provide functions similar to C++ STL

• list
• vector
• deque
• queue

Canonical GTFS Validator project for schedule (static) files.

• gtfs-dataset
• gtfs-validator
• gtfs-zip
• gtfs-feed

go-etl is a toolset for data extraction, transformation and loading.

• etl
• mysql
• go
• postgresql

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.

A set of .NET projection libraries

• event-sourcing
• c-sharp
• sql
• projections

Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.

A set of open-source .NET libraries helping with DDD (domain-driven design) and CQRS (command query responsibility segregation).

• ddd
• cqrs
• nhibernate
• asp-net-core

hypermatch is a high-performance ⚡ Go library designed for rapid matching of a large number of rules to events. It processes thousands of events per second 🚀 against extensive rule sets in-memory with minimal latency ⏱️.

• go
• golang
• matching
• matching-engine

Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.

• csv
• data-engineering
• data-quality
• diff

GTFS Data Pipeline for TfNSW Bus Datasets

• gtfs
• gtfs-static
• gtfs-realtime
• tfnsw

Estimating the number of clusters in a data set via the gap statistic. Implemented in H2O-3

• kmeans
• kmeans-clustering
• clustering
• gap-statistic