Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Repository Index
47,133 repositories indexed
1–13 of 13 · Most stars
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
The open source and enterprise-ready webhooks service 🦀
- webhooks
- rust
- webhook-server
- webhook-service
Object-storage native graph database with branching and typed ontology
- apache-arrow
- datafusion
- graph-database
- knowledge-graph
Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- data-engineering
- data-integration
- data-pipeline
- data-quality
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
- change-data-capture
- data
- data-collection
- data-engineering
A robust web archive analytics toolkit
- python
- web
- warc
- bigdata
Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.
- analytics
- feature-store
- fraud-detection
- python
Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency
provides a Suricata Eve output for Kafka with Suricate Eve plugin
- suricata
- intrusion-detection