Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
132,241 visible repositories in the index
1–15 of 15 · Most stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Graph-Native Semantic Layer for Trustworthy Enterprise AI
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Incremental engine for long horizon agents 🌟 Star if you like it!
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
A high-performance Postgres replication engine written in Rust. Embed it in your Rust application or run it as a standalone binary.
Markdown 批量导出工具、开放式跨平台博客解决方案,随意组合写作平台(语雀/Notion/FlowUs/飞书/我来Wolai)和博客平台(Hexo/Vitepress/Halo/Confluence/WordPress等)
cryo is the easiest way to extract blockchain data to parquet, csv, json, or python dataframes
Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 437 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
Open Source research tool to search, browse, analyze and explore large document collections by Semantic Search Engine and Open Source Text Mining & Text Analytics platform (Integrates ETL for document processing, OCR for images & PDF, named entity recognition for persons, organizations & locations, metadata management by thesaurus & ontologies, search user interface & search apps for fulltext search, faceted search & knowledge graph)
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
Blockchain Indexing Engine