Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–15 of 15 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Python SQL Parser and Transpiler
- transpiler
- sql
- python
- parser
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
🌊 Online machine learning in Python
- incremental-learning
- machine-learning
- python
- online-learning
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
WebSocket and WAMP in Python for Twisted and asyncio
- python
- websocket
- real-time
- wamp
Python Streaming DataFrames for Kafka
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
MLBox is a powerful Automated Machine Learning python library.
- machine-learning
- auto-ml
- kaggle
- deep-learning
Real-time stream processing for python
- streaming-data
- python
- async
- real-time
The world's cleanest AutoML library ✨ - Do hyperparameter tuning with the right pipeline abstractions to write clean deep learning production pipelines. Let your pipeline steps have hyperparameter spaces. Design steps in your pipeline like components. Compatible with Scikit-Learn, TensorFlow, and most other libraries, frameworks and MLOps environments.
- pipeline
- pipeline-framework
- machine-learning
- deep-learning
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
A micro workers class that easily manages a pool of threads to optimise simultaneous jobs and jobs endings, for MicroPython (used on Pycom modules & ESP32)
- pycom
- micropython
- lopy
- wipy