Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–21 of 21 · Most stars
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
- gbdt
- gbrt
- gbm
- distributed-systems
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
- notebooks
- python
- data-science
- machine-learning
JavaScript framework for visual programming
- visual-programming
- node-editor
- dataflow-programming
- graph-editor
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Simple flow library 🖥️🖱️
- flow
- javascript
- javascript-library
- flow-based-programming
Code security scanning tool (SAST) to discover, filter and prioritize security and privacy risks.
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Dataflow compiler for QNN inference on FPGAs
Spreadsheet-like dataflow programming in ClojureScript.
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
A lightweight, header-only, high-performance reactive programming framework for C++20.
- modern-cpp
- template-meta-programming
- mvvm-framework
- ui-dataflow
Streaming reactive and dataflow graphs in Python
- python
- python3
- stream
- data-pipeline
A Common Lisp implementation of the dataflow programming paradigm
Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.
Easier-than-print dataflow tracing to tap> and Portal with automatic last-input function replay on eval, instant re-render and effortless extraction of traced data
Flowgraph package for scalable asynchronous system development
- dataflow
- scalable
- distributed
ETL scripts for EOS.
- eos
- eosio
- etl
- apache-beam
Upipe dataflow framework
Send OTP verification messages via DataFlows SMS API for Supabase Auth
- australia
- authentication
- edge-functions
- otp