Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
142,273 visible repositories in the index
1–12 of 12 · Most stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
A system for agentic LLM-powered data processing and ETL
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Python Fast Dataflow programming framework for Data pipeline work( Web Crawler,Machine Learning,Quantitative Trading.etc)
Input pipeline framework
The data transformation tool the open-source community deserves.
ETL scripts for EOS.