Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–24 of 76 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
A realtime distributed messaging platform
Premium Queue package for handling distributed jobs and messages in NodeJS.
Apache Doris is a real-time analytics and hybrid search database for AI agents.
- olap
- database
- hudi
- iceberg
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
An asynchronous event driven PHP socket framework. Supports HTTP, Websocket, SSL and other custom protocols.
- websocket
- timer
- socket
- event-driven
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
Python SQL Parser and Transpiler
- transpiler
- sql
- python
- parser
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
Highly scalable realtime pub/sub and RPC framework
🌊 Online machine learning in Python
- incremental-learning
- machine-learning
- python
- online-learning
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
State of the Art Natural Language Processing
Queueing jobs in Postgres from Node.js like a boss
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
High performance, distributed and low latency publish-subscribe platform.
- pubsub
- networking
- emitter
- mqtt
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native