Trending repositories

1.crmne / zapfast

ZapFast: a fast, native WhatsApp client in Rust and egui

• chat
• desktop-app
• egui
• linux
• macos
• messaging
• +4 more

2.stepll2026 / OntiCards

TranslatedOntiCards is an enterprise-oriented AI Data Hub that offers one-stop capabilities around business databases: data source connection, metadata understanding, natural language query, cross-source fusion, data quality inspection, query auditing, data asset management, and more

3.OpenDCAI / DataFlow

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

• data
• data-cleaning
• data-pipelines
• data-processing
• data-science
• data-synthesis
• +7 more

4.kestra-io / kestra

Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

• orchestration
• data-orchestration
• high-availability
• infrastructure-as-code
• automation
• devops
• +14 more

5.apache / spark

Apache Spark - A unified analytics engine for large-scale data processing

• python
• scala
• r
• java
• big-data
• jdbc
• +2 more

6.open-metadata / OpenMetadata

The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.

• metadata
• datadiscovery
• dataquality
• data-profiling
• metadata-management
• data-catalog
• +14 more

7.apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

• airflow
• apache
• apache-airflow
• python
• scheduler
• workflow
• +14 more

8.ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

• ray
• distributed
• parallel
• machine-learning
• reinforcement-learning
• deep-learning
• +14 more

9.cocoindex-io / cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

• ai
• change-data-capture
• data-indexing
• etl
• indexing
• python
• +14 more

10.nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.

• go
• golang
• messaging
• message-bus
• message-queue
• cloud-native
• +9 more

11.Unstructured-IO / unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

• deep-learning
• document-parsing
• machine-learning
• nlp
• ocr
• information-retrieval
• +14 more

12.hugues31 / duckpad

The data transformation tool the open-source community deserves.

• data-engineering
• duckdb

13.debezium / debezium

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

• change-data-capture
• kafka-connect
• apache-kafka
• debezium
• cdc
• database
• +5 more

14.airbytehq / airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering
• java
• python
• +14 more

15.dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

• data
• python
• data-engineering
• data-lake
• data-loading
• data-warehouse
• +4 more

16.the-momentum / open-wearables

Self-hosted platform to unify wearable health data through one AI-ready API.

• wearables
• apple-health
• apple-watch
• fitness
• fitness-analysis
• fitness-api
• +9 more

17.eclipse-mosquitto / mosquitto

Eclipse Mosquitto - An open source MQTT broker

• mqtt
• broker
• mosquitto
• eclipse-iot
• iot
• messaging
• +3 more

18.apache / iceberg

Apache Iceberg

• iceberg
• apache
• hacktoberfest

19.dbt-labs / dbt

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

• dbt-viewpoint
• slack
• pypa
• data-modeling
• business-intelligence
• analytics
• +1 more

20.thingsboard / thingsboard

All-in-one IoT Platform - Device management, data collection, processing and visualization.

• visualization
• iot
• iot-platform
• mqtt
• middleware
• thingsboard
• +12 more

21.dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.

• data-pipelines
• dagster
• workflow
• data-science
• workflow-automation
• python
• +9 more

22.evidentlyai / evidently

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

• data-drift
• jupyter-notebook
• pandas-dataframe
• machine-learning
• model-monitoring
• html-report
• +8 more

23.PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

• python
• workflow
• data-engineering
• data-science
• workflow-engine
• prefect
• +8 more

24.openimsdk / open-im-server

IM Chat OpenClaw

• im
• chat
• messaging
• messenger
• openclaw

25.redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

• kafka
• cpp
• kubernetes
• go
• streaming
• redpanda
• +7 more

26.bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models
• large-language-models
• machine-learning
• +14 more

27.emqx / emqx

The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles

• mqtt
• iot
• mqtt-broker
• erlang
• iot-middleware
• broker
• +14 more

28.centrifugal / centrifugo

Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.

• websocket
• redis
• real-time
• messaging
• scalability
• eventsource
• +14 more

29.apache / seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

• data-integration
• high-performance
• offline
• real-time
• apache
• batch
• +8 more

30.apache / nifi

Apache NiFi

• nifi
• java
• hacktoberfest
• apache

31.pgmq / pgmq

A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.

• message-queue

32.slothflowlabs / duckle

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.

• data-engineering
• data-integration
• data-pipeline
• data-quality
• duckdb
• elt
• +14 more

33.apache / iggy

Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed

• messaging
• quic
• rust
• streaming
• http
• iggy
• +2 more

34.Netflix / metaflow

Build, Manage and Deploy AI/ML Systems

• machine-learning
• model-management
• ai
• ml
• ml-platform
• ml-infrastructure
• +14 more

35.aeron-io / aeron

Efficient reliable UDP unicast, UDP multicast, and IPC message transport

• messaging
• java
• c-plus-plus
• ipc
• multicast-streams
• c

36.alicevision / Meshroom

Node-based Visual Programming Toolbox

• photogrammetry
• computer-vision
• 3d-reconstruction
• alicevision
• structure-from-motion
• multi-view-stereo
• +8 more

37.apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

• arrow
• big-data
• dataframe
• olap
• python
• query-engine
• +3 more

38.apache / flink

Apache Flink

• scala
• java
• big-data
• flink
• python
• sql

39.apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

• workflow-schedule
• azkaban
• airflow
• task-scheduler
• job-scheduler
• cloud-native
• +5 more

40.zeromq / libzmq

ZeroMQ core engine in C++, implements ZMTP/3.1

• zeromq
• zmq
• messaging
• concurrency
• pubsub
• pushpull
• +5 more

41.ThreeDotsLabs / watermill

Building event-driven applications the easy way in Go.

• watermill
• event-driven
• event-sourcing
• sagas
• events
• stream-processing
• +7 more

42.apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

• datalake
• lakehouse
• metadata
• federated-query
• stratosphere
• metalake
• +5 more

43.alibaba / DataX

TranslatedDataX is the open-source version of Alibaba Cloud DataWorks Data Integration.

44.fivetran / great_expectations

Always know what to expect from your data.

• pipeline-tests
• dataquality
• datacleaning
• datacleaner
• data-science
• data-profiling
• +14 more

45.unionai-oss / pandera

A light-weight, flexible, and expressive statistical data testing library

• pandas
• validation
• schema
• dataframes
• testing
• pandas-validation
• +12 more

46.bruin-data / bruin

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

• analytics
• bigquery
• data-modeling
• data-pipelines
• python
• snowflake
• +5 more

47.cleanlab / cleanlab

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

• weak-supervision
• data-cleaning
• data-quality
• noisy-labels
• data-centric-ai
• out-of-distribution-detection
• +14 more

48.apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam
• batch
• golang
• +2 more

49.hazelcast / hazelcast

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

• java
• hazelcast
• in-memory
• big-data
• scalability
• distributed
• +9 more

50.svix / svix-webhooks

The open source and enterprise-ready webhooks service 🦀

• webhooks
• rust
• webhook-server
• webhook-service
• webhook
• webhooks-server
• +13 more

51.KristianP26 / ble-scale-sync

Universal BLE Smart Scale bridge. Captures body composition from Renpho, Xiaomi & 20+ others, syncs to Garmin Connect, Strava, MQTT (Home Assistant), InfluxDB, Webhooks, Ntfy & local files (CSV/JSONL). Headless CLI for Raspberry Pi, Linux, macOS & Windows.

• garmin
• typescript
• mqtt
• ble
• body-composition
• garmin-connect
• +14 more

52.rudderlabs / rudder-server

Privacy and Security focused Segment-alternative, in Golang and React

• privacy
• warehouse-management
• data-warehouse
• customer-data
• customer-data-pipeline
• customer-data-platform
• +14 more

53.bruin-data / ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

• bigquery
• copy-database
• data-ingestion
• data-integration
• data-pipeline
• duckdb
• +4 more

54.mochi-mqtt / server

The fully compliant, embeddable high-performance Go MQTT v5 server for IoT, smarthome, and pubsub

• mqtt
• mqtt-broker
• mqtt-server
• mqtt-smarthome
• golang
• iot
• +7 more

55.emirpasic / gods

GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more

• go
• golang
• data-structure
• map
• tree
• set
• +10 more

56.apache / hadoop

Apache Hadoop

• hadoop

57.tinode / chat

Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots

• chat
• chat-application
• chatapp
• go
• golang
• instant-messaging
• +14 more

58.kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

• kedro
• hacktoberfest
• mlops
• python
• machine-learning
• machine-learning-engineering
• +3 more

59.dimitri / pgloader

Migrate to PostgreSQL in a single command!

• postgresql
• migration
• mysql
• mssql
• sqlite
• common-lisp
• +7 more

60.apache / camel

Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.

• camel
• integration
• java
• microservices
• cloud-native
• data-transformation
• +10 more

61.zenml-io / zenml

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

• mlops
• machine-learning
• data-science
• production-ready
• devops-tools
• zenml
• +14 more

62.onetimesecret / onetimesecret

Keep passwords and other sensitive information out of your chat logs and inboxes.

• privacy
• onetime
• secrets
• chat
• email
• messaging
• +1 more

63.meltano / meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source
• opensource
• data
• +14 more

64.confluentinc / librdkafka

The Apache Kafka C/C++ library

• kafka
• kafka-consumer
• apache-kafka
• high-performance
• c
• librdkafka
• +3 more

65.alibaba / canal

TranslatedAlibaba's MySQL binlog incremental subscription & consumption component

66.dask / dask

Parallel computing with task scheduling

• dask
• python
• pydata
• numpy
• pandas
• scikit-learn
• +1 more

67.cloudquery / cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

• aws
• gcp
• azure
• sql
• data-integration
• elt
• +14 more

68.ag2ai / faststream

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

• asyncio
• kafka
• python
• rabbitmq
• asyncapi
• distributed-systems
• +7 more

69.ArroyoSystems / arroyo

Distributed stream processing engine in Rust

• data
• dev-tools
• infrastructure
• kafka
• rust
• sql
• +3 more

70.ankane / pgsync

Sync data from one Postgres database to another

• postgresql

71.BrighterCommand / Brighter

A framework for building messaging apps with .NET and C#.

• command-processor
• command-dispatcher
• brighter
• asynchronously
• nuget-packages
• scale
• +11 more

72.timeplus-io / proton

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

• clickhouse
• kakfa
• sql
• redpanda
• single-binary
• stream-processing
• +10 more

73.bytewax / bytewax

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering
• data-processing
• data-science
• +3 more

74.rabbitmq / rabbitmq-server

Open source RabbitMQ: core server and tier 1 (built-in) plugins

• rabbitmq
• amqp
• messaging
• amqp-0-9-1
• amqp1-0
• mqtt
• +4 more

75.delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

• spark
• acid
• big-data
• analytics
• delta-lake

76.pentaho / pentaho-kettle

Pentaho Data Integration ( ETL ) a.k.a Kettle

77.feast-dev / feast

The Open Source Feature Store for AI/ML

• machine-learning
• features
• ml
• big-data
• feature-store
• python
• +4 more

78.dotnetcore / CAP

Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

• kafka
• rabbitmq
• distributed-transactions
• outbox-pattern
• eventbus
• microservices
• +1 more

79.mahmoud / boltons

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

• python
• utilities
• cache
• data-structures
• statistics
• data-science
• +5 more

80.apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.

• hudi
• apachehudi
• datalake
• bigdata
• apachespark
• incremental-processing
• +3 more

81.dunglas / mercure

🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications

• api
• web-api
• server-sent-events
• subscriptions
• golang
• hypermedia
• +10 more

82.asyncapi / spec

The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.

• asyncapi
• reactive
• specification
• websockets
• amqp
• mqtt
• +2 more

83.meetfranz / franz

Franz is a free messaging app for services like WhatsApp, Slack, Messenger and many more.

• messaging
• electron
• franz
• messenger
• whatsapp
• slack
• +3 more

84.PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

• etl
• sql
• bigquery
• cloud-native
• distributed-systems
• kafka
• +10 more

85.nanomq / nanomq

An ultra-lightweight and blazing-fast Messaging Bus/MQTT Broker for Edge & SDV

• mqtt
• mqtt-broker
• iot-edge
• nng
• aio
• actor-model
• +14 more

86.apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

• spark
• hive
• sql
• thrift
• jdbc
• spark-sql
• +4 more

87.lf-edge / ekuiper

Lightweight data stream processing engine for IoT edge

• edge
• sql
• iot
• edge-analytics
• edge-streaming
• edge-computing
• +2 more

88.narwhals-dev / narwhals

Lightweight and extensible compatibility layer between dataframe libraries!

• cudf
• ibis
• pandas
• polars
• pyarrow
• dask
• +2 more

89.Desbordante / desbordante-core

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

• data-analytics
• data-cleaning
• data-cleansing
• data-engineering
• data-exploration
• data-mining
• +14 more

90.apache / rocketmq

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

• rocketmq
• cloud-native
• eventing
• java
• messaging
• streaming
• +1 more

91.Data-Centric-AI-Community / fg-data-profiling

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

• pandas-profiling
• pandas-dataframe
• statistics
• jupyter-notebook
• exploration
• data-science
• +14 more

92.edenhill / kcat

Generic command line non-JVM Apache Kafka producer and consumer

93.go-mysql-org / go-mysql

a powerful mysql toolset with Go

94.RoaringBitmap / RoaringBitmap

A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others

• java
• bitset
• roaring-bitmaps
• roaringbitmap
• druid
• spark
• +1 more

95.vernemq / vernemq

A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!

• mqtt
• m2m
• scalable
• broker
• erlang
• distributed
• +13 more

96.Wisser / Jailer

Database Subsetting and Relational Data Browsing Tool.

• database
• subsetting
• frontend
• export
• extract
• sql
• +13 more

97.frain-dev / convoy

The Cloud Native Webhooks Gateway

• webhooks
• convoy
• golang
• automation
• cli
• cloud
• +12 more

98.apache / hamilton

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

• data-science
• python
• dag
• data-engineering
• dataframe
• etl
• +13 more

99.moscajs / aedes

Barebone MQTT broker that can run on any stream server, the node way

• mqtt
• mqtt-broker
• nodejs
• mqtt-server
• mosquitto
• iot
• +3 more

100.dask / distributed

A distributed task scheduler for Dask

• pydata
• dask
• distributed-computing
• python