Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Incremental engine for long horizon agents 🌟 Star if you like it!

• ai
• change-data-capture
• data-indexing
• etl

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

• data
• data-cleaning
• data-pipelines
• data-processing

A light-weight, flexible, and expressive statistical data testing library

• pandas
• validation
• schema
• dataframes

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

• machine-learning
• natural-language-processing
• tensorflow
• deep-learning

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering

A pure Python implementation of Apache Spark's RDD and DStream interfaces.

• data-science
• data-processing
• python
• apache-spark

Public tracker for Scramjet Cloud Platform, a platform that bring data from many environments together.

• data-stream
• event-stream
• transformations
• reactive-programming