1.OpenDCAI / DataFlow
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
- data-science
- data-synthesis
- +7 more
+5,113 stars
from 3,320 · score 8.54
Trending repositories
Ranked by Attention Score: stars gained over the last 180 complete days
1–100 of 190 by Attention Score · May 8, 2026 – Nov 3, 2026 (UTC)
2.kestra-io / kestra
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
- orchestration
- data-orchestration
- high-availability
- infrastructure-as-code
- automation
- devops
- +14 more
+3,690 stars
from 26,452 · score 8.21
3.cocoindex-io / cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
- indexing
- python
- +14 more
+3,004 stars
from 8,814 · score 8.01
4.google-ai-edge / mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
- mediapipe
- c-plus-plus
- computer-vision
- deep-learning
- android
- video-processing
- +11 more
+2,638 stars
from 34,901 · score 7.88
5.apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
+2,312 stars
from 45,043 · score 7.75
6.open-metadata / OpenMetadata
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
- metadata-management
- data-catalog
- +14 more
+2,211 stars
from 13,461 · score 7.70
7.ray-project / ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
+2,018 stars
from 42,177 · score 7.61
8.PrefectHQ / prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
- python
- workflow
- data-engineering
- data-science
- workflow-engine
- prefect
- +8 more
+1,943 stars
from 22,164 · score 7.57
9.apache / kafka
Apache Kafka - A distributed event streaming platform
+1,824 stars
from 32,297 · score 7.51
10.dbt-labs / dbt
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
- dbt-viewpoint
- slack
- pypa
- data-modeling
- business-intelligence
- analytics
- +1 more
+1,470 stars
from 12,646 · score 7.29
11.apache / spark
Apache Spark - A unified analytics engine for large-scale data processing
+1,398 stars
from 43,008 · score 7.24
12.nats-io / nats-server
High-Performance server for NATS.io, the cloud and edge native messaging system.
- go
- golang
- messaging
- message-bus
- message-queue
- cloud-native
- +9 more
+1,353 stars
from 19,641 · score 7.21
13.AutoMQ / automq
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
+1,246 stars
from 9,675 · score 7.13
14.Unstructured-IO / unstructured
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
- ocr
- information-retrieval
- +14 more
+1,116 stars
from 14,561 · score 7.02
15.apache / iggy
Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed
+1,070 stars
from 4,165 · score 6.98
16.dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
- workflow-automation
- python
- +9 more
+1,020 stars
from 15,345 · score 6.93
17.dlt-hub / dlt
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
- data
- python
- data-engineering
- data-lake
- data-loading
- data-warehouse
- +4 more
+784 stars
from 5,267 · score 6.67
18.centrifugal / centrifugo
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
+749 stars
from 10,197 · score 6.62
19.debezium / debezium
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
- cdc
- database
- +5 more
+685 stars
from 12,627 · score 6.53
+644 stars
from 8,760 · score 6.47
21.evidentlyai / evidently
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
- model-monitoring
- html-report
- +8 more
+610 stars
from 7,424 · score 6.42
+608 stars
from 25,835 · score 6.41
23.openimsdk / open-im-server
IM Chat OpenClaw
+559 stars
from 16,193 · score 6.33
24.apache / seatunnel
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
- apache
- batch
- +8 more
+534 stars
from 9,251 · score 6.28
25.fivetran / great_expectations
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
- data-science
- data-profiling
- +14 more
+513 stars
from 11,401 · score 6.24
26.apache / gravitino
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
- stratosphere
- metalake
- +5 more
+482 stars
from 2,864 · score 6.18
27.risingwavelabs / risingwave
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
- database
- stream-processing
- rust
- postgresql
- kafka
- materialized-view
- +5 more
+458 stars
from 8,939 · score 6.13
28.numaproj / numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
- pipeline
- map-reduce
- +1 more
+386 stars
from 2,451 · score 5.96
29.feast-dev / feast
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
- feature-store
- python
- +4 more
+383 stars
from 6,970 · score 5.95
30.aeron-io / aeron
Efficient reliable UDP unicast, UDP multicast, and IPC message transport
- messaging
- java
- c-plus-plus
- ipc
- multicast-streams
- c
+373 stars
from 8,575 · score 5.92
31.apache / dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
- workflow-schedule
- azkaban
- airflow
- task-scheduler
- job-scheduler
- cloud-native
- +5 more
+361 stars
from 14,189 · score 5.89
32.apache / rocketmq
Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
- rocketmq
- cloud-native
- eventing
- java
- messaging
- streaming
- +1 more
+345 stars
from 22,304 · score 5.85
33.provectus / kafka-ui
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
- kafka-client
- opensource
- +12 more
+340 stars
from 11,974 · score 5.83
34.delta-io / delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
+339 stars
from 8,727 · score 5.83
35.cleanlab / cleanlab
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
- data-centric-ai
- out-of-distribution-detection
- +14 more
+331 stars
from 11,398 · score 5.81
36.Netflix / metaflow
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
- ml-platform
- ml-infrastructure
- +14 more
+307 stars
from 10,031 · score 5.73
37.ag2ai / faststream
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
- asyncio
- kafka
- python
- rabbitmq
- asyncapi
- distributed-systems
- +7 more
+278 stars
from 5,113 · score 5.63
+268 stars
from 15,441 · score 5.59
39.nats-io / nats.go
Golang client for NATS, the cloud native messaging system.
- go
- golang
- nats
- microservices
- microservices-architecture
- pub-sub
- +3 more
+254 stars
from 6,543 · score 5.54
40.zenml-io / zenml
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
- devops-tools
- zenml
- +14 more
+254 stars
from 5,381 · score 5.54
+245 stars
from 6,046 · score 5.51
42.bruin-data / bruin
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
- python
- snowflake
- +5 more
+233 stars
from 1,566 · score 5.46
43.alibaba / canal
TranslatedAlibaba's MySQL binlog incremental subscription & consumption component
+232 stars
from 29,512 · score 5.45
44.apache / hudi
Upserts, Deletes And Incremental Processing on Big Data.
- hudi
- apachehudi
- datalake
- bigdata
- apachespark
- incremental-processing
- +3 more
+229 stars
from 6,119 · score 5.44
45.apache / camel
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices
- cloud-native
- data-transformation
- +10 more
+225 stars
from 6,165 · score 5.42
46.apache / pulsar
Apache Pulsar - distributed pub-sub messaging system
- pulsar
- pubsub
- messaging
- streaming
- queuing
- event-streaming
+220 stars
from 15,132 · score 5.40
47.ArroyoSystems / arroyo
Distributed stream processing engine in Rust
- data
- dev-tools
- infrastructure
- kafka
- rust
- sql
- +3 more
+204 stars
from 4,873 · score 5.32
48.apache / shardingsphere
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
+202 stars
from 20,615 · score 5.31
49.asyncapi / spec
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
- asyncapi
- reactive
- specification
- websockets
- amqp
- mqtt
- +2 more
+187 stars
from 5,147 · score 5.24
50.apache / beam
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
- batch
- golang
- +2 more
+176 stars
from 8,529 · score 5.18
51.apache / hamilton
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
- dataframe
- etl
- +13 more
+153 stars
from 2,469 · score 5.04
52.eclipse-iceoryx / iceoryx
Eclipse iceoryx™ - true zero-copy inter-process-communication
- iceoryx
- eclipse
- ipc
- inter-process-communication
- cpp
- zero-copy
- +10 more
+151 stars
from 2,055 · score 5.02
53.apache / datafusion-ballista
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
- python
- query-engine
- +3 more
+150 stars
from 2,016 · score 5.02
54.spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
- hadoop
+138 stars
from 18,644 · score 4.93
55.apache / incubator-kie
Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.
- rule-engine
- java
- dmn
- hacktoberfest
- apache
- drools
- +3 more
+119 stars
from 6,222 · score 4.79
56.softwaremill / elasticmq
In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.
- scala
- sqs-interface
- elasticmq
- amazon-sqs
- aws
- aws-sqs
- +1 more
+119 stars
from 2,834 · score 4.79
57.cloudamqp / lavinmq
Ultra quick message queue and streaming server
- message-queue
- server
- amqp
- background-jobs
- job-queue
- message-broker
- +3 more
+117 stars
from 914 · score 4.77
58.MassTransit / MassTransit
Distributed Application Framework for .NET
+112 stars
from 7,701 · score 4.73
59.RoaringBitmap / RoaringBitmap
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
- java
- bitset
- roaring-bitmaps
- roaringbitmap
- druid
- spark
- +1 more
+110 stars
from 3,846 · score 4.71
60.fluvio-community / fluvio
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
- serverless
- stateful
- +13 more
+108 stars
from 5,152 · score 4.69
61.apache / sedona
A cluster computing framework for processing large-scale geospatial data
- cluster-computing
- spatial-sql
- geospatial
- spatial-analysis
- spatial-query
- scala
- +3 more
+105 stars
from 2,321 · score 4.66
62.OpenHFT / Chronicle-Queue
Micro second messaging that stores everything to disk
- chronicle
- queue
- java
- performance
- low-latency
- persistance
+102 stars
from 3,725 · score 4.63
63.timeplus-io / proton
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
- single-binary
- stream-processing
- +10 more
+100 stars
from 2,181 · score 4.62
64.thingsboard / thingsboard-gateway
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
+99 stars
from 2,106 · score 4.61
65.beava-dev / beava
Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.
- analytics
- feature-store
- fraud-detection
- python
- real-time
- rust
- +4 more
+93 stars
from 46 · score 4.54
66.bytewax / bytewax
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
- data-processing
- data-science
- +3 more
+91 stars
from 1,979 · score 4.52
67.reductstore / reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
- blob-storage
- storage-engine
- storage
- historian
- timeseries
- time-series
- +9 more
+90 stars
from 287 · score 4.51
68.vernemq / vernemq
A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!
- mqtt
- m2m
- scalable
- broker
- erlang
- distributed
- +13 more
+89 stars
from 3,557 · score 4.50
69.absmach / magistrala
IoT Platform Framework
- golang
- iot-platform
- edge-computing
- distributed-systems
- iot-cloud
- iot-gateway
- +14 more
+84 stars
from 2,574 · score 4.44
70.projectnessie / nessie
Nessie: Transactional Catalog for Data Lakes with Git-like semantics
- data
- spark
- java
- iceberg
- aws-lambda
- git
+81 stars
from 1,445 · score 4.41
71.Wisser / Jailer
Database Subsetting and Relational Data Browsing Tool.
+77 stars
from 3,139 · score 4.36
72.meetfranz / franz
Franz is a free messaging app for services like WhatsApp, Slack, Messenger and many more.
+72 stars
from 4,657 · score 4.29
73.lf-edge / ekuiper
Lightweight data stream processing engine for IoT edge
+67 stars
from 1,680 · score 4.22
74.apache / kyuubi
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
+66 stars
from 2,310 · score 4.20
75.bloomberg / blazingmq
A modern high-performance open source message queuing system
+62 stars
from 3,159 · score 4.14
76.awslabs / deequ
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
+61 stars
from 3,596 · score 4.13
77.snowplow / snowplow
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
- data-collection
- product-analytics
- +3 more
+60 stars
from 6,988 · score 4.11
+59 stars
from 1,210 · score 4.09
+57 stars
from 1,014 · score 4.06
80.samber / ro
🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications
- event-driven
- generics
- go
- golang
- pipe
- pipeline
- +11 more
+55 stars
from 641 · score 4.03
81.thingsboard / tbmq
The ultimate distributed MQTT broker. Handles 100M+ connections and 10M msg/sec with ease. Built on Kafka to provide industrial-grade persistence and eliminate data loss.
+54 stars
from 722 · score 4.01
82.elixir-broadway / broadway
Concurrent and multi-stage data ingestion and data processing with Elixir
- elixir
- genstage
- data-ingestion
- data-processing
- concurrent
- broadway
+53 stars
from 2,639 · score 3.99
83.arkflow-rs / arkflow
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
- sql
- datafusion
- +14 more
+52 stars
from 1,264 · score 3.97
84.kroxylicious / kroxylicious
Kroxylicious, the snappy open source proxy for Apache Kafka®
+51 stars
from 281 · score 3.95
85.polyaxon / polyaxon
AI Infra / AI Orchestration / AI Control Plane
+50 stars
from 3,697 · score 3.93
86.apache / ignite
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
- ignite
- data-management-platform
- +9 more
+49 stars
from 5,041 · score 3.91
87.FluidGroup / swiftui-messaging-ui
For SwiftUI apps - a primitive component to make Chat-UI with stable prepending - no scroll jumps when loading older messages.
+45 stars
from 44 · score 3.83
+44 stars
from 2,418 · score 3.81
89.graphframes / graphframes
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
- dataframes
- graphs
- +8 more
+43 stars
from 1,169 · score 3.78
90.WeBankFinTech / DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
+42 stars
from 3,233 · score 3.76
91.quixio / quix-streams
Python Streaming DataFrames for Kafka
+42 stars
from 1,540 · score 3.76
92.apache / streampipes
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
- iot
- analytics
- edge
- stream-processing
- iiot
- self-service
- +6 more
+40 stars
from 716 · score 3.71
93.yahoo / CMAK
CMAK is a tool for managing Apache Kafka clusters
+39 stars
from 11,879 · score 3.69
+38 stars
from 6,661 · score 3.66
95.apache / eventmesh
EventMesh is a new generation serverless event middleware for building distributed event-driven applications.
- esb
- event-driven
- event-mesh
- event-governance
- cloud-native
- serverless
- +11 more
+38 stars
from 1,721 · score 3.66
+37 stars
from 3,334 · score 3.64
+35 stars
from 2,428 · score 3.58
98.aws / aws-sdk-pandas
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
- python
- aws
- pandas
- apache-arrow
- apache-parquet
- data-engineering
- +14 more
+32 stars
from 4,090 · score 3.50
99.Aiven-Open / karapace
Karapace - Your Apache Kafka® essentials in one tool
- kafka
- schema-registry
- rest-proxy
+32 stars
from 609 · score 3.50
100.capitalone / datacompy
Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!
- python
- pandas
- spark
- data
- data-science
- compare
- +8 more
+31 stars
from 634 · score 3.47