An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
- asyncapi
- reactive
- specification
- websockets
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
Eclipse iceoryx™ - true zero-copy inter-process-communication
- iceoryx
- eclipse
- ipc
- inter-process-communication
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
The type-safe, event-driven workflow orchestration library that scales with your business. Build robust, distributed workflows in Go with compile-time safety, automatic retries, and horizontal scaling out of the box. Integrate with your existing tech stack using adapters 🚀
- eventdrivenarchitecture
- golang
- kafka
- reflex
Elixir Adapter for EventStore
- eventstore
- elixir-lang
- cqrs
- erlang
Formal specification and generation of verifiable binary parsers, message generators and protocol state machines
A message queue framework, with support for middleware and multiple adapters.
- elixir
- message-queue
General-purpose, formally-verified, 64-bit operating system in SPARK/Ada for x86-64
A secure and high performances microkernel for building secure MCU-based IoTs
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
- dotnet
- dotnet-core
- etl
- etl-framework
Apache Kafka Ada client library, based off of librdkafka
ada privilege escalation
SPARK TOTP library
Production-grade Server-Sent Events (SSE) for Fiber v3. Event coalescing, priority lanes, topic wildcards, adaptive throttling, built-in auth, Prometheus metrics, graceful drain.
- event-streaming
- fasthttp
- fiber
- gofiber