Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Fancy stream processing made operationally mundane

• message-queue
• stream-processing
• streaming-data
• message-bus

Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.

• camel
• integration
• java
• microservices

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

• etl
• sql
• bigquery
• cloud-native

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

• flow
• rust
• rust-lang
• stream

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

• go
• data-integration
• etl
• data-pipeline

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)

• cdc
• tidb
• mysql
• kafka

High performance Kafka consumer for InfluxDB. Supports collectd message formats.

• kafka-influxdb
• kafka-consumer
• logstash
• influxdb