Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

• data-analysis
• data-science
• large-language-models
• llm

A system for agentic LLM-powered data processing and ETL

• data
• etl
• llm
• python

Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.

• machine-learning
• convolutional-networks
• embedding-vectors
• embeddings

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

• claude-code
• codex
• data-processing
• harness-engineering

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

Code to download, process, and analyse the Online Encyclopedia of Integer Sequences