Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
Full-stack .Net 10 Clean Architecture (Microservices, Modular Monolith, Monolith), Blazor, Angular 22, React 19, Vue 3.5, BFF with YARP, NextJs 16, Domain-Driven Design, CQRS, SOLID, Asp.Net Core Identity Custom Storage, OpenID Connect, EF Core, OpenTelemetry, SignalR, Background Services, Health Checks, Rate Limiting, Clouds (Azure, AWS, GCP), ...
Pluggable foundation blocks for building distributed apps.
- azure
- c-sharp
- lock
- message-bus
Object-storage native graph database with branching and typed ontology
- apache-arrow
- datafusion
- graph-database
- knowledge-graph
A RocksDB compliant high performance scalable embedded key-value store
- key-value-store
- kvs
- storage-engine
- speedb
Event store for Azure Table Storage
High-performance, time-indexed object storage for robotics and industrial IoT
- blob-storage
- storage-engine
- storage
- historian
A Simplest in-memory and Distributed Storage-Agnostic Message Queue for Your Concurrent Go Program
- concurrency
- go
- goroutine
- priority-queue
Time-series database and pub/sub messaging server for IoT and real-time devices, in Go: an embeddable storage engine with TTLs, wildcard topics and encryption, and a clustered server over gRPC, TCP and WebSocket with replication and rolling upgrades.
A simplified (in memory) example of Event Sourcing implementation for banking domain.
- event-sourcing
- ddd
- ddd-sample
Fast implementation of queue in OCaml
- queue
- ocaml
- gadt
Pipeline de ingestão e busca semântica de documentos para RAG — com rate limiting, circuit breaker e fila resiliente.