Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

• data
• data-cleaning
• data-pipelines
• data-processing

A light-weight, flexible, and expressive statistical data testing library

• pandas
• validation
• schema
• dataframes

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

• machine-learning
• natural-language-processing
• tensorflow
• deep-learning

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

A pure Python implementation of Apache Spark's RDD and DStream interfaces.

• data-science
• data-processing
• python
• apache-spark