Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–24 of 72 · Most stars
Apache Kafka - A distributed event streaming platform
A realtime distributed messaging platform
Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
- rocketmq
- cloud-native
- eventing
- java
All-in-one IoT Platform - Device management, data collection, processing and visualization.
- visualization
- iot
- iot-platform
- mqtt
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
- workflow-schedule
- azkaban
- airflow
- task-scheduler
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
- chat
- chat-application
- chatapp
- go
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
High performance, distributed and low latency publish-subscribe platform.
- pubsub
- networking
- emitter
- mqtt
Micro second messaging that stores everything to disk
- chronicle
- queue
- java
- performance
LFX Community Data Platform (CDP)
- community
- cdp
- open-source
Database Subsetting and Relational Data Browsing Tool.
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s