47,133 repositories indexed
1–22 of 22 · Most stars
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
Fancy stream processing made operationally mundane
The leader in Customer Data Infrastructure
🍴 Web applications made easy. Since 2011.
Migrate to PostgreSQL in a single command!
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Embulk: Pluggable Bulk Data Loader.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
The ultimate distributed MQTT broker. Handles 100M+ connections and 10M msg/sec with ease. Built on Kafka to provide industrial-grade persistence and eliminate data loss.
Amazon Redshift Database Loader implemented in AWS Lambda
🚃 A library for handling mediator patterns and simplified CQRS patterns within an event-driven architecture, inspired by csharp MediatR library.
STOMP client for Embarcadero Delphi and FreePascal.
Time-series database and pub/sub messaging server for IoT and real-time devices, in Go: an embeddable storage engine with TTLs, wildcard topics and encryption, and a clustered server over gRPC, TCP and WebSocket with replication and rolling upgrades.
A blazing fast fully-automated CSV to database importer
Tuple MapReduce for Hadoop: Hadoop API made easy
Track Go benchmark performance over time by storing results in InfluxDB
Stream Processing Made Easy
A production-grade messaging abstraction for Go. It provides a unified API to decouple business logic from various brokers including Kafka, RabbitMQ, RocketMQ, NATS, Redis, and AWS SQS. Featuring built-in OpenTelemetry integration and engineered for high reliability.
Stateful actors on Apache Flink 2.x and Java 21 — durable per-key state, exactly-once messaging, Kafka and Kinesis I/O, Kubernetes-native deployment. Continues the Apache Stateful Functions programming model on the modern Flink line.
Open, audit-grade agentic data quality framework with portable industry packs
Production-grade Server-Sent Events (SSE) for Fiber v3. Event coalescing, priority lanes, topic wildcards, adaptive throttling, built-in auth, Prometheus metrics, graceful drain.