Sign in

Tags /

Data pipelines and messaging

Stream processing, ETL, message queues, and event systems.

Repositories

791

Stars, in total now

3.5M

Net stars gained, last 7 days

+4,803

-6% from the 7 days before

New Admissions, last 7 days

1

Net stars gained each day, last 90 days

Most in one day: +1,430 on Aug 10, 2026

Aug 5, 2026Nov 2, 2026

How a repository gets this Tag

This is a top-level Tag: it groups repositories by area. When an agent gave a repository finer Tags from its README, the repository gets the top-level Tags above them. Until then, a rule in the AwesomeRepos source code gives it this Tag when one of these is true. A machine applies the rule to each repository each hour, and a repository can have more than one Tag. A Tag does not rank repositories and does not say that one is better than another.

One of its topics is airflow, apache-flink, apache-kafka, apache-spark, big-data, bigdata, cqrs, data-engineering, data-lake, data-pipeline, dbt, elt, etl, event-driven, event-sourcing, flink, hadoop, kafka, lakehouse, message-broker, message-queue, messaging, nats, pipeline, pub-sub, pubsub, pyspark, queue, rabbitmq, spark, or one of 2 more topics.

A topic starts with kafka-.

Its description contains the words “etl”, “data pipeline”, “data pipelines”, “stream processing”, “message queue”, “message broker”, “kafka”, “event sourcing”, “apache spark”.

Withheld Repositories are left out of all numbers on this page. The daily line adds up the Star History of the repositories that have the Tag now.