Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.
- scala
- sqs-interface
- elasticmq
- amazon-sqs
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
- flogo
- iot
- integration
- golang
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
Nessie: Transactional Catalog for Data Lakes with Git-like semantics
- data
- spark
- java
- iceberg
Amazon Redshift Database Loader implemented in AWS Lambda
AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.
- aws-cli
- athena-cli
- aws-athena
- python
:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Build and deploy a serverless data pipeline on AWS with no effort.
- aws-cdk
- data-pipeline
- stepfunctions
- glue
Use AWS CDK with CodePipeline to deploy a TypeScript Create-React-App.
- aws-cdk
- aws-codepipeline
- cra-pipeline
- react
A production-grade messaging abstraction for Go. It provides a unified API to decouple business logic from various brokers including Kafka, RabbitMQ, RocketMQ, NATS, Redis, and AWS SQS. Featuring built-in OpenTelemetry integration and engineered for high reliability.
AWS_MSK_IAM Authentication Plugin for Broadway Kafka