1.airbytehq / airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
- data
- pipeline
- data-analysis
- data-engineering
- java
- python
- +14 more
Ranked by Attention Score: stars gained over the last 180 complete days
60 repositories by Attention Score · May 8, 2026 – Nov 3, 2026 (UTC)
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
All-in-one IoT Platform - Device management, data collection, processing and visualization.
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Eclipse Mosquitto - An open source MQTT broker
Node-based Visual Programming Toolbox
Open source RabbitMQ: core server and tier 1 (built-in) plugins
TranslatedDataX is the open-source version of Alibaba Cloud DataWorks Data Integration.
lakeFS - Data version control for your data lake | Git for data
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
Logstash - transport and process your logs, events, or other data
Migrate to PostgreSQL in a single command!
GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Privacy and Security focused Segment-alternative, in Golang and React
The Apache Kafka C/C++ library
Pentaho Data Integration ( ETL ) a.k.a Kettle
Real-time monitor and web admin for Celery distributed task queue
The Cloud Native Webhooks Gateway
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
Healthcare data infrastructure for next generation care delivery.
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
visual data prep powered by python
Ncurses based XMPP client
The swiss army knife of healthcare integration.
Generic command line non-JVM Apache Kafka producer and consumer
:bus: Simple and lean service bus implementation for .NET
Confluent Schema Registry for Kafka
Modular, Production-Ready, Open-Source EHR
A 100% native C# implementation of ZeroMQ for .NET
Maxwell's daemon, a mysql-to-json kafka producer
MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilise the resources of multiple clustered machines.
Ruby and Rails efficient Kafka processing framework
RabbitMQ .NET client for .NET Standard 2.0+ and .NET 4.6.2+
An implementation of Actor, Publish-Subscribe, and CSP models in one rather small C++ framework. With performance, quality, and stability proved by years in the production.
Python Stream Processing. A Faust fork
Compositional, streaming I/O library for Scala
Python Stream Processing
Fast and reliable message broker built on top of Kafka.
The database purpose-built for stream processing applications.
Bunny is a popular, easy to use, mature Ruby client for RabbitMQ
A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.
ENIGMA HALFpipe is a user-friendly software that facilitates reproducible analysis of fMRI data
Crossbar.io - WAMP application router
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
JSON-RPC 2.0 implementation over WebSockets for Node.js and JavaScript/TypeScript
Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
Distributed Stream and Batch Processing
CogComp's Natural Language Processing Libraries and Demos: Modules include lemmatizer, ner, pos, prep-srl, quantifier, question type, relation-extraction, similarity, temporal normalizer, tokenizer, transliteration, verb-sense, and more.
A C# implementation of WAMP (The Web Application Messaging Protocol)
Ballerina MQTT Module.