Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
59,132 visible repositories in the index
1–24 of 31 · Most stars
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
- asyncapi
- reactive
- specification
- websockets
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
A network event stream processing system, in Clojure.
- riemann
- events
- clojure
- monitoring
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
- machine-learning
- ml
- model-validation
- data-validation
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
LFX Community Data Platform (CDP)
- community
- cdp
- open-source
Ruby and Rails efficient Kafka processing framework
- karafka-framework
- kafka
- kafka-client
- kafka-clients
Ncurses based XMPP client
- xmpp-client
- c
- xmpp
- chat
Clean APIs for data cleaning. Python implementation of R package Janitor
- pandas
- dataframe
- data
- cleaning-data
HiveMQ CE is a Java-based open source MQTT broker that fully supports MQTT 3.x and MQTT 5. It is the foundation of the HiveMQ Enterprise Connectivity and Messaging Platform
Data validation toolkit for assessing and monitoring data quality.
- data-validation
- tabular-data
- data-testing
- data-quality
The data platform for manufacturing
- oee
- iot
The data transformation tool the open-source community deserves.
- data-engineering
- duckdb
- data-lineage
- data-pipeline
Lightweight and Extensible Infrastructure for Building Web Applications - Web Application Framework
- dotnet-core
- efcore
- aspnet-core
- solid
Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.
StreamFlow™ is a stream processing tool designed to help build and monitor processing workflows.
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
- data-ops
- data-quality
- data-quality-checks
- data-quality-measurement
Daggy - Data Aggregation Utility and C/C++ developer library for data streams catching
- serverless-framework
- streaming
- stream-processing
- qt