Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Repository Index
47,133 repositories indexed
1–18 of 18 · Most stars
Incremental engine for long horizon agents 🌟 Star if you like it!
A collection of handy Bash One-Liners and terminal tricks for data processing and Linux system maintenance.
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
Unified querying, transformation, and modification of JSON, TOML, YAML, XML, INI, HCL, KDL and CSV.
A light-weight, flexible, and expressive statistical data testing library
Kubernetes-native platform to run massively parallel data/streaming jobs
Concurrent and multi-stage data ingestion and data processing with Elixir
Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/
Python Stream Processing
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
👾~ music, eternal ~ 👾
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
Public tracker for Scramjet Cloud Platform, a platform that bring data from many environments together.
Read and write Excel (.xlsx, .xls), CSV, and ODS files in Java with one line of code. Annotation-based mapping, SAX streaming, built-in validation, 19 type converters, cell styling. A high-performance alternative to Apache POI.
Data policy IN, dynamic view OUT: PACE is the Policy As Code Engine. It helps you to programatically create and apply a data policy to a processing platform like Databricks, Snowflake or BigQuery (or plain 'ol Postgres, even!) with definitions imported from Collibra, Datahub, ODD and the like.
High performant ETL engine written in TypeScript