Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
Logstash - transport and process your logs, events, or other data
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Python Streaming DataFrames for Kafka
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.
- postgres
- postgresql
- python
- infectious-disease
Real-time GTFS database loader and ORM library.