Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–24 of 118 · Most stars
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
Apache Kafka - A distributed event streaming platform
TranslatedA repository covering six columns: Spring Boot 2.X, Spring Cloud, Spring Cloud Alibaba, Dubbo, distributed message queues, and distributed transactions. I hope you will give it a quick tap and click Star in the upper right corner. Thank you, 1024
- spring-boot
- spring-cloud
- spring-cloud-alibaba
- dubbo
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
- telegraf
- monitoring
- time-series
- metrics
TranslatedBig Data Beginner's Guide :star:
- hadoop
- hdfs
- yarn
- mapreduce
Translatedflink learning blog. http://www.54tianzhisheng.cn/ Covers Flink introduction, concepts, principles, hands-on practice, performance tuning, source code analysis, and more. Includes learning examples involving Flink Connector, Metrics, Library, DataStream API, Table API & SQL, and also shares large-scale project cases of Flink in production (PV/UV, log storage, real-time deduplication of 10 billion records, monitoring and alerting). You are welcome to support my column 《大数据实时计算引擎 Flink 实战与性能优化》 ("Big Data Real-Time Computing Engine Flink: Hands-On Practice and Performance Optimization")
Translated[Big Tech Interview Column] A technical guide for Java programmers, with interview questions, system architecture, career tips, mainstream middleware, and more, to help you become an even better version of yourself!
- java
- spring
- springboot
- dubbo
C++ Parallel Computing and Asynchronous Networking Framework
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
Sarama is a Go library for Apache Kafka.
- kafka
- kafka-client
- go
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
CMAK is a tool for managing Apache Kafka clusters
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
TranslatedFocused on big data learning and interviews, the road to becoming a big data god begins. Flink/Spark/Hadoop/Hbase/Hive...
Building event-driven applications the easy way in Go.
- watermill
- event-driven
- event-sourcing
- sagas
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
Dockerfile for Apache Kafka
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
- camel
- integration
- java
- microservices