Trending repositories

1.rustfs / rustfs

RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.

• bigdata
• cloud-native
• filesystem
• object-storage
• rust
• s3
• +6 more

2.ModernRelay / omnigraph

Object-storage native graph database with branching and typed ontology

• apache-arrow
• datafusion
• graph-database
• knowledge-graph
• lakehouse
• lance
• +5 more

3.apache / iggy

Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed

• messaging
• quic
• rust
• streaming
• http
• iggy
• +2 more

4.cocoindex-io / cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

• ai
• change-data-capture
• data-indexing
• etl
• indexing
• python
• +14 more

5.slothflowlabs / duckle

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.

• data-engineering
• data-integration
• data-pipeline
• data-quality
• duckdb
• elt
• +14 more

6.dbt-labs / dbt

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

• dbt-viewpoint
• slack
• pypa
• data-modeling
• business-intelligence
• analytics
• +1 more

7.vectordotdev / vector

A high-performance observability data pipeline.

• logs
• metrics
• observability
• forwarder
• events
• stream-processing
• +12 more

8.restatedev / restate

Restate is the platform for building resilient applications that tolerate all infrastructure faults w/o the need for a PhD.

• distributed-systems
• async-await
• durable-execution
• microservices
• consistent-state
• reliable-communication
• +4 more

9.quickwit-oss / quickwit

Cloud-native OSS search engine for observability

• rust
• log-management
• logs
• tantivy
• cloud-native
• open-source
• +4 more

10.risingwavelabs / risingwave

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql
• kafka
• materialized-view
• +5 more

11.svix / svix-webhooks

The open source and enterprise-ready webhooks service 🦀

• webhooks
• rust
• webhook-server
• webhook-service
• webhook
• webhooks-server
• +13 more

12.ArroyoSystems / arroyo

Distributed stream processing engine in Rust

• data
• dev-tools
• infrastructure
• kafka
• rust
• sql
• +3 more

13.dathere / qsv

Blazing-fast, Neuro-symbolic Data-Wrangling toolkit

• csv
• data-wrangling
• opendata
• data-engineering
• ckan
• excel
• +14 more

14.PRQL / prql

PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement

• data
• pipeline
• sql

15.apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

• arrow
• big-data
• dataframe
• olap
• python
• query-engine
• +3 more

16.MaterializeInc / materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

• rust
• database
• sql
• streaming
• kafka
• distributed-systems
• +11 more

17.fluvio-community / fluvio

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time
• serverless
• stateful
• +13 more

18.estuary / flow

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊

• change-data-capture
• data
• data-collection
• data-engineering
• data-integration
• data-pipeline
• +2 more

19.numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs

• kubernetes
• stream-processing
• data-processing
• k8s
• pipeline
• map-reduce
• +1 more

20.matanolabs / matano

Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS

• aws
• cloud
• security
• big-data
• serverless
• apache-iceberg
• +14 more

21.arkflow-rs / arkflow

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

• flow
• rust
• rust-lang
• stream
• sql
• datafusion
• +14 more

22.build-trust / ockam

Orchestrate end-to-end encryption, cryptographic identities, mutual authentication, and authorization policies between distributed applications – at massive scale.

• messaging
• authentication
• key-management
• identity
• trust
• kafka
• +12 more

23.reductstore / reductstore

High-performance, time-indexed object storage for robotics and industrial IoT

• blob-storage
• storage-engine
• storage
• historian
• timeseries
• time-series
• +9 more

24.laminardb / laminardb

Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency

25.chatnoir-eu / chatnoir-resiliparse

A robust web archive analytics toolkit

• python
• web
• warc
• bigdata
• extraction
• webarchive
• +2 more

26.beava-dev / beava

Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.

• analytics
• feature-store
• fraud-detection
• python
• real-time
• rust
• +4 more