Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
132,241 visible repositories in the index
1–13 of 13 · Most stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Incremental engine for long horizon agents 🌟 Star if you like it!
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
Scalable data pre processing and curation toolkit for LLMs
Extract Transform Load for Python 3.5+
Python Streaming DataFrames for Kafka
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Concurrent Python made simple
visual data prep powered by python
Open source project for data preparation for GenAI applications
The data transformation tool the open-source community deserves.
High performant ETL engine written in TypeScript