Proxy based Redis cluster solution supporting pipeline and scaling dynamically
Repository Index
47,133 repositories indexed
1–24 of 24 · Most stars
🌌 A complete search engine and RAG pipeline in your browser, server or edge network with support for full-text, vector, and hybrid search in less than 2kb.
- data-structures
- full-text
- search
- typo-tolerance
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
Where data access meets operational intelligence
- mysql
- postgresql
- ai-agent
- data-catalog
Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
- cron
- workflow-engine
- workflow-scheduler
- workflow-orchestration
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Surging is a micro-service engine that provides a lightweight, high-performance, modular RPC request pipeline. support Event-based Asynchronous Pattern and reactive programming.
- microservices
- apigateway
- surging
- rabbitmq
A fast Golang Redis client that supports Client Side Caching, Auto Pipelining, RDMA, etc.
- go
- golang
- redis
- redis-client
High-performance time-series aggregation for PostgreSQL
- realtime
- stream-processing
- analytics
- sql
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
A lightweight stream processing library for Go
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)
Twitter Image Pipeline is a robust and performant image loading and caching framework for iOS clients
- image-pipeline
- ios
- swift
- objective-c
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- data-engineering
- data-integration
- data-pipeline
- data-quality
Fast, pipelined, resilient Redis driver for Elixir. 🛍
- redis
- redis-sentinel
- redis-pubsub
- redis-client
zerocode-tdd is a community-developed, free, open-source, outcome-driven automated testing framework for Data Pipelines, ETL, REST API, Kafka(Data Streams), Databases and Load scenarios. all defined in simple JSON or YAML — with zero coding.
- soap
- assertions
- java
- api
ChEMBL database structure pipelines
:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Golang Mongodb Pagination for official mongodb/mongo-go-driver package which supports both normal queries and Aggregation pipelines with all information like Total records, Page, Per Page, Previous, Next, Total Page and query results.
- go-mongo
- mongo-go-pagination
- mongo-go-driver
- mongo-pagination
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
Data ingestion engine — Singer taps to Parquet in cloud buckets