ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Repository Index
47,133 repositories indexed
49–72 of 224 · Most stars
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Distributed stream processing engine in Rust
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
A light-weight, flexible, and expressive statistical data testing library
- pandas
- validation
- schema
- dataframes
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
- machine-learning
- ml
- model-validation
- data-validation
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
AI Infra / AI Orchestration / AI Control Plane
Source-agnostic distributed change data capture system
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
Sync data from one Postgres database to another
Utils for streaming large files (S3, HDFS, gzip, bz2...)
- python
- s3
- hdfs
- webhdfs
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
LFX Community Data Platform (CDP)
- community
- cdp
- open-source
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
- workflow
- governance
- azkaban
- davinci
Database Subsetting and Relational Data Browsing Tool.
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s