Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–17 of 17 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
Clean APIs for data cleaning. Python implementation of R package Janitor
- pandas
- dataframe
- data
- cleaning-data
Jupyter magics and kernels for working with remote Spark clusters
Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.
- pandas
- pandas-summary
- dataframes
- data-science
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
- anomaly-detection
- bigquery
- cli
- data-engineering