Trending repositories

1.cocoindex-io / cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

• ai
• change-data-capture
• data-indexing
• etl
• indexing
• python
• +14 more

2.ModernRelay / omnigraph

Object-storage native graph database with branching and typed ontology

• apache-arrow
• datafusion
• graph-database
• knowledge-graph
• lakehouse
• lance
• +5 more

3.dbt-labs / dbt

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

• dbt-viewpoint
• slack
• pypa
• data-modeling
• business-intelligence
• analytics
• +1 more

4.apache / iggy

Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed

• messaging
• quic
• rust
• streaming
• http
• iggy
• +2 more

5.risingwavelabs / risingwave

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql
• kafka
• materialized-view
• +5 more

6.numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs

• kubernetes
• stream-processing
• data-processing
• k8s
• pipeline
• map-reduce
• +1 more

7.svix / svix-webhooks

The open source and enterprise-ready webhooks service 🦀

• webhooks
• rust
• webhook-server
• webhook-service
• webhook
• webhooks-server
• +13 more

8.ArroyoSystems / arroyo

Distributed stream processing engine in Rust

• data
• dev-tools
• infrastructure
• kafka
• rust
• sql
• +3 more

9.dathere / qsv

Blazing-fast, Neuro-symbolic Data-Wrangling toolkit

• csv
• data-wrangling
• opendata
• data-engineering
• ckan
• excel
• +14 more

10.apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

• arrow
• big-data
• dataframe
• olap
• python
• query-engine
• +3 more

11.MaterializeInc / materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

• rust
• database
• sql
• streaming
• kafka
• distributed-systems
• +11 more

12.fluvio-community / fluvio

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time
• serverless
• stateful
• +13 more

13.beava-dev / beava

Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.

• analytics
• feature-store
• fraud-detection
• python
• real-time
• rust
• +4 more

14.reductstore / reductstore

High-performance, time-indexed object storage for robotics and industrial IoT

• blob-storage
• storage-engine
• storage
• historian
• timeseries
• time-series
• +9 more

15.estuary / flow

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊

• change-data-capture
• data
• data-collection
• data-engineering
• data-integration
• data-pipeline
• +2 more

16.arkflow-rs / arkflow

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

• flow
• rust
• rust-lang
• stream
• sql
• datafusion
• +14 more

17.matanolabs / matano

Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS

• aws
• cloud
• security
• big-data
• serverless
• apache-iceberg
• +14 more

18.laminardb / laminardb

Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency

19.chatnoir-eu / chatnoir-resiliparse

A robust web archive analytics toolkit

• python
• web
• warc
• bigdata
• extraction
• webarchive
• +2 more