Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
87,075 visible repositories in the index
1–9 of 9 · Most stars
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
A system for agentic LLM-powered data processing and ETL
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
- machine-learning
- convolutional-networks
- embedding-vectors
- embeddings
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
- claude-code
- codex
- data-processing
- harness-engineering
Scalable data pre processing and curation toolkit for LLMs
- data-curation
- llm
- data
- data-prep
Extract Transform Load for Python 3.5+
- data-processing
- extract-transform-load
- bonobo
- python3
Python Streaming DataFrames for Kafka