Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–24 of 27 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
- kedro
- hacktoberfest
- mlops
- python
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Elyra extends JupyterLab with an AI centric approach.
- notebooks
- notebook-jupyter
- jupyterlab
- jupyterlab-extensions
A library for event sourcing in Python.
- domain-driven-design
- ddd
- eventsourcing
- cqrs
asyncio client for kafka
- asyncio
- kafka
- kafka-client
Python3 client for NATS
- asyncio
- nats
- python3
- cloud-native
OntiCards 是面向企业的 AI 数据中枢（AI Data Hub），围绕业务数据库提供一站式能力：数据源连接、元数据理解、自然语言查询、跨源融合、数据质检、查询审计与数据资产管理等
Karapace - Your Apache Kafka® essentials in one tool
- kafka
- schema-registry
- rest-proxy
The idiomatic asyncio MQTT client
Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.
- pandas
- pandas-summary
- dataframes
- data-science
The repository host gives no description.
- hacktoberfest
- data
- datalineage
- dbt
Async MQTT client for CPython and MicroPython with asyncio support.
- async-python
- asyncio
- embedded
- embedded-systems
Declarative data quality engine. Define checks in YAML, run anywhere.
- airflow
- anomaly-detection
- data-contracts
- data-quality