Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–24 of 30 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
- kedro
- hacktoberfest
- mlops
- python
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
A light-weight, flexible, and expressive statistical data testing library
- pandas
- validation
- schema
- dataframes
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
- machine-learning
- ml
- model-validation
- data-validation
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
asyncio client for kafka
- asyncio
- kafka
- kafka-client
Python3 client for NATS
- asyncio
- nats
- python3
- cloud-native
TranslatedOntiCards is an enterprise-oriented AI Data Hub that offers one-stop capabilities around business databases: data source connection, metadata understanding, natural language query, cross-source fusion, data quality inspection, query auditing, data asset management, and more
Karapace - Your Apache Kafka® essentials in one tool
- kafka
- schema-registry
- rest-proxy
The idiomatic asyncio MQTT client
Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.
- pandas
- pandas-summary
- dataframes
- data-science