1.OpenDCAI / DataFlow
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
- data-science
- data-synthesis
- +7 more
+5,113 stars
from 3,320 · score 8.54
Trending repositories
Ranked by Attention Score: stars gained over the last 180 complete days
1–100 of 393 by Attention Score · May 8, 2026 – Nov 3, 2026 (UTC)
2.kestra-io / kestra
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
- orchestration
- data-orchestration
- high-availability
- infrastructure-as-code
- automation
- devops
- +14 more
+3,690 stars
from 26,452 · score 8.21
3.cocoindex-io / cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
- indexing
- python
- +14 more
+3,004 stars
from 8,814 · score 8.01
4.google-ai-edge / mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
- mediapipe
- c-plus-plus
- computer-vision
- deep-learning
- android
- video-processing
- +11 more
+2,638 stars
from 34,901 · score 7.88
5.apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
+2,312 stars
from 45,043 · score 7.75
6.open-metadata / OpenMetadata
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
- metadata-management
- data-catalog
- +14 more
+2,211 stars
from 13,461 · score 7.70
7.ray-project / ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
+2,018 stars
from 42,177 · score 7.61
8.PrefectHQ / prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
- python
- workflow
- data-engineering
- data-science
- workflow-engine
- prefect
- +8 more
+1,943 stars
from 22,164 · score 7.57
9.apache / kafka
Apache Kafka - A distributed event streaming platform
+1,824 stars
from 32,297 · score 7.51
10.dbt-labs / dbt
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
- dbt-viewpoint
- slack
- pypa
- data-modeling
- business-intelligence
- analytics
- +1 more
+1,470 stars
from 12,646 · score 7.29
11.apache / spark
Apache Spark - A unified analytics engine for large-scale data processing
+1,398 stars
from 43,008 · score 7.24
12.nats-io / nats-server
High-Performance server for NATS.io, the cloud and edge native messaging system.
- go
- golang
- messaging
- message-bus
- message-queue
- cloud-native
- +9 more
+1,353 stars
from 19,641 · score 7.21
13.AutoMQ / automq
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
+1,246 stars
from 9,675 · score 7.13
14.airbytehq / airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
- data
- pipeline
- data-analysis
- data-engineering
- java
- python
- +14 more
+1,223 stars
from 21,089 · score 7.11
15.the-momentum / open-wearables
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
- fitness-analysis
- fitness-api
- +9 more
+1,174 stars
from 1,579 · score 7.07
16.Unstructured-IO / unstructured
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
- ocr
- information-retrieval
- +14 more
+1,116 stars
from 14,561 · score 7.02
17.thingsboard / thingsboard
All-in-one IoT Platform - Device management, data collection, processing and visualization.
- visualization
- iot
- iot-platform
- mqtt
- middleware
- thingsboard
- +12 more
+1,083 stars
from 21,550 · score 6.99
18.apache / iggy
Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed
+1,070 stars
from 4,165 · score 6.98
19.dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
- workflow-automation
- python
- +9 more
+1,020 stars
from 15,345 · score 6.93
20.pathwaycom / pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
+844 stars
from 61,177 · score 6.74
21.dlt-hub / dlt
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
- data
- python
- data-engineering
- data-lake
- data-loading
- data-warehouse
- +4 more
+784 stars
from 5,267 · score 6.67
22.centrifugal / centrifugo
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
+749 stars
from 10,197 · score 6.62
23.emqx / emqx
The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles
+737 stars
from 16,143 · score 6.60
24.debezium / debezium
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
- cdc
- database
- +5 more
+685 stars
from 12,627 · score 6.53
25.redpanda-data / redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
- streaming
- redpanda
- +7 more
+680 stars
from 11,980 · score 6.52
+644 stars
from 8,760 · score 6.47
27.bigscience-workshop / petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
- large-language-models
- machine-learning
- +14 more
+640 stars
from 10,048 · score 6.46
28.evidentlyai / evidently
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
- model-monitoring
- html-report
- +8 more
+610 stars
from 7,424 · score 6.42
+608 stars
from 25,835 · score 6.41
30.bruin-data / ingestr
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
- data-pipeline
- duckdb
- +4 more
+596 stars
from 3,443 · score 6.39
31.pgmq / pgmq
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
+562 stars
from 4,811 · score 6.33
32.openimsdk / open-im-server
IM Chat OpenClaw
+559 stars
from 16,193 · score 6.33
33.apache / seatunnel
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
- apache
- batch
- +8 more
+534 stars
from 9,251 · score 6.28
34.eclipse-mosquitto / mosquitto
Eclipse Mosquitto - An open source MQTT broker
+532 stars
from 10,796 · score 6.28
35.fivetran / great_expectations
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
- data-science
- data-profiling
- +14 more
+513 stars
from 11,401 · score 6.24
36.apache / gravitino
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
- stratosphere
- metalake
- +5 more
+482 stars
from 2,864 · score 6.18
37.risingwavelabs / risingwave
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
- database
- stream-processing
- rust
- postgresql
- kafka
- materialized-view
- +5 more
+458 stars
from 8,939 · score 6.13
38.alicevision / Meshroom
Node-based Visual Programming Toolbox
- photogrammetry
- computer-vision
- 3d-reconstruction
- alicevision
- structure-from-motion
- multi-view-stereo
- +8 more
+404 stars
from 12,658 · score 6.00
39.rabbitmq / rabbitmq-server
Open source RabbitMQ: core server and tier 1 (built-in) plugins
+395 stars
from 13,559 · score 5.98
40.numaproj / numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
- pipeline
- map-reduce
- +1 more
+386 stars
from 2,451 · score 5.96
41.feast-dev / feast
The Open Source Feature Store for AI/ML
- machine-learning
- features
- ml
- big-data
- feature-store
- python
- +4 more
+383 stars
from 6,970 · score 5.95
42.aeron-io / aeron
Efficient reliable UDP unicast, UDP multicast, and IPC message transport
- messaging
- java
- c-plus-plus
- ipc
- multicast-streams
- c
+373 stars
from 8,575 · score 5.92
43.apache / dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
- workflow-schedule
- azkaban
- airflow
- task-scheduler
- job-scheduler
- cloud-native
- +5 more
+361 stars
from 14,189 · score 5.89
44.apache / rocketmq
Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
- rocketmq
- cloud-native
- eventing
- java
- messaging
- streaming
- +1 more
+345 stars
from 22,304 · score 5.85
45.tinode / chat
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
- chat
- chat-application
- chatapp
- go
- golang
- instant-messaging
- +14 more
+344 stars
from 13,204 · score 5.84
46.provectus / kafka-ui
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
- kafka-client
- opensource
- +12 more
+340 stars
from 11,974 · score 5.83
47.delta-io / delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
+339 stars
from 8,727 · score 5.83
48.cleanlab / cleanlab
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
- data-centric-ai
- out-of-distribution-detection
- +14 more
+331 stars
from 11,398 · score 5.81
49.Netflix / metaflow
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
- ml-platform
- ml-infrastructure
- +14 more
+307 stars
from 10,031 · score 5.73
50.ThreeDotsLabs / watermill
Building event-driven applications the easy way in Go.
- watermill
- event-driven
- event-sourcing
- sagas
- events
- stream-processing
- +7 more
+298 stars
from 9,641 · score 5.70
51.svix / svix-webhooks
The open source and enterprise-ready webhooks service 🦀
- webhooks
- rust
- webhook-server
- webhook-service
- webhook
- webhooks-server
- +13 more
+291 stars
from 3,185 · score 5.68
52.alibaba / DataX
TranslatedDataX is the open-source version of Alibaba Cloud DataWorks Data Integration.
+287 stars
from 17,091 · score 5.66
53.Data-Centric-AI-Community / fg-data-profiling
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
- pandas-profiling
- pandas-dataframe
- statistics
- jupyter-notebook
- exploration
- data-science
- +14 more
+284 stars
from 13,463 · score 5.65
54.ag2ai / faststream
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
- asyncio
- kafka
- python
- rabbitmq
- asyncapi
- distributed-systems
- +7 more
+278 stars
from 5,113 · score 5.63
+268 stars
from 15,441 · score 5.59
56.nats-io / nats.go
Golang client for NATS, the cloud native messaging system.
- go
- golang
- nats
- microservices
- microservices-architecture
- pub-sub
- +3 more
+254 stars
from 6,543 · score 5.54
57.zenml-io / zenml
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
- devops-tools
- zenml
- +14 more
+254 stars
from 5,381 · score 5.54
+245 stars
from 6,046 · score 5.51
59.treeverse / lakeFS
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
- data-lake
- aws-s3
- +13 more
+244 stars
from 5,331 · score 5.50
60.kedro-org / kedro
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
- kedro
- hacktoberfest
- mlops
- python
- machine-learning
- machine-learning-engineering
- +3 more
+241 stars
from 10,801 · score 5.49
61.PeerDB-io / peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
- distributed-systems
- kafka
- +10 more
+237 stars
from 3,076 · score 5.47
62.bruin-data / bruin
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
- python
- snowflake
- +5 more
+233 stars
from 1,566 · score 5.46
63.alibaba / canal
TranslatedAlibaba's MySQL binlog incremental subscription & consumption component
+232 stars
from 29,512 · score 5.45
64.apache / hudi
Upserts, Deletes And Incremental Processing on Big Data.
- hudi
- apachehudi
- datalake
- bigdata
- apachespark
- incremental-processing
- +3 more
+229 stars
from 6,119 · score 5.44
65.zeromq / libzmq
ZeroMQ core engine in C++, implements ZMTP/3.1
- zeromq
- zmq
- messaging
- concurrency
- pubsub
- pushpull
- +5 more
+228 stars
from 10,822 · score 5.43
66.apache / camel
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices
- cloud-native
- data-transformation
- +10 more
+225 stars
from 6,165 · score 5.42
67.turbot / steampipe
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
- security
- aws
- +14 more
+222 stars
from 7,772 · score 5.41
68.apache / pulsar
Apache Pulsar - distributed pub-sub messaging system
- pulsar
- pubsub
- messaging
- streaming
- queuing
- event-streaming
+220 stars
from 15,132 · score 5.40
69.meltano / meltano
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
- opensource
- data
- +14 more
+208 stars
from 2,467 · score 5.34
70.onetimesecret / onetimesecret
Keep passwords and other sensitive information out of your chat logs and inboxes.
+206 stars
from 2,776 · score 5.33
71.ArroyoSystems / arroyo
Distributed stream processing engine in Rust
- data
- dev-tools
- infrastructure
- kafka
- rust
- sql
- +3 more
+204 stars
from 4,873 · score 5.32
72.dathere / qsv
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
- ckan
- excel
- +14 more
+204 stars
from 3,614 · score 5.32
73.apache / shardingsphere
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
+202 stars
from 20,615 · score 5.31
74.unionai-oss / pandera
A light-weight, flexible, and expressive statistical data testing library
+201 stars
from 4,307 · score 5.31
75.elastic / logstash
Logstash - transport and process your logs, events, or other data
+187 stars
from 14,784 · score 5.24
76.asyncapi / spec
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
- asyncapi
- reactive
- specification
- websockets
- amqp
- mqtt
- +2 more
+187 stars
from 5,147 · score 5.24
77.nanomq / nanomq
An ultra-lightweight and blazing-fast Messaging Bus/MQTT Broker for Edge & SDV
- mqtt
- mqtt-broker
- iot-edge
- nng
- aio
- actor-model
- +14 more
+187 stars
from 2,468 · score 5.24
+186 stars
from 13,756 · score 5.23
79.apache / beam
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
- batch
- golang
- +2 more
+176 stars
from 8,529 · score 5.18
80.dimitri / pgloader
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
- sqlite
- common-lisp
- +7 more
+176 stars
from 6,379 · score 5.18
81.cloudquery / cloudquery
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
+174 stars
from 6,375 · score 5.16
82.redpanda-data / connect
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
- logs
- stream-processor
- +11 more
+166 stars
from 8,627 · score 5.12
83.nsqio / nsq
A realtime distributed messaging platform
+153 stars
from 25,621 · score 5.04
84.apache / hamilton
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
- dataframe
- etl
- +13 more
+153 stars
from 2,469 · score 5.04
85.eclipse-iceoryx / iceoryx
Eclipse iceoryx™ - true zero-copy inter-process-communication
- iceoryx
- eclipse
- ipc
- inter-process-communication
- cpp
- zero-copy
- +10 more
+151 stars
from 2,055 · score 5.02
86.apache / datafusion-ballista
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
- python
- query-engine
- +3 more
+150 stars
from 2,016 · score 5.02
87.emirpasic / gods
GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more
- go
- golang
- data-structure
- map
- tree
- set
- +10 more
+147 stars
from 17,309 · score 5.00
88.narwhals-dev / narwhals
Lightweight and extensible compatibility layer between dataframe libraries!
+145 stars
from 1,601 · score 4.98
89.dunglas / mercure
🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications
- api
- web-api
- server-sent-events
- subscriptions
- golang
- hypermedia
- +10 more
+141 stars
from 5,216 · score 4.96
90.MaterializeInc / materialize
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
- rust
- database
- sql
- streaming
- kafka
- distributed-systems
- +11 more
+140 stars
from 6,252 · score 4.95
91.spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
- hadoop
+138 stars
from 18,644 · score 4.93
+135 stars
from 12,388 · score 4.91
93.rudderlabs / rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
- customer-data-pipeline
- customer-data-platform
- +14 more
+135 stars
from 4,380 · score 4.91
94.confluentinc / librdkafka
The Apache Kafka C/C++ library
- kafka
- kafka-consumer
- apache-kafka
- high-performance
- c
- librdkafka
- +3 more
+123 stars
from 960 · score 4.82
95.pentaho / pentaho-kettle
Pentaho Data Integration ( ETL ) a.k.a Kettle
+119 stars
from 8,293 · score 4.79
96.apache / incubator-kie
Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.
- rule-engine
- java
- dmn
- hacktoberfest
- apache
- drools
- +3 more
+119 stars
from 6,222 · score 4.79
97.softwaremill / elasticmq
In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.
- scala
- sqs-interface
- elasticmq
- amazon-sqs
- aws
- aws-sqs
- +1 more
+119 stars
from 2,834 · score 4.79
98.cloudamqp / lavinmq
Ultra quick message queue and streaming server
- message-queue
- server
- amqp
- background-jobs
- job-queue
- message-broker
- +3 more
+117 stars
from 914 · score 4.77
99.MassTransit / MassTransit
Distributed Application Framework for .NET
+112 stars
from 7,701 · score 4.73
100.RoaringBitmap / RoaringBitmap
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
- java
- bitset
- roaring-bitmaps
- roaringbitmap
- druid
- spark
- +1 more
+110 stars
from 3,846 · score 4.71