1.rustfs / rustfs
RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
Ranked by Attention Score: stars gained over the last 30 complete days
1–100 of 469 by Attention Score · Oct 4, 2026 – Nov 2, 2026 (UTC)
RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
ZapFast: a fast, native WhatsApp client in Rust and egui
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
bluetooth mesh chat, IRC vibes
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
Object-storage native graph database with branching and typed ontology
For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
Send push notifications to your phone or desktop using PUT/POST
This is a repo with links to everything you'd ever want to learn about data engineering
ClickHouse® is a real-time analytics database management system
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
Apache Superset is a Data Visualization and Data Exploration Platform
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
Cross-platform, customizable ML solutions for live and streaming media.
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Apache Spark - A unified analytics engine for large-scale data processing
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
Learn how to develop, deploy and iterate on production-grade ML applications.
Apache Kafka - A distributed event streaming platform
mall项目是一套电商系统,包括前台商城系统及后台管理系统,基于Spring Boot+MyBatis实现,采用Docker容器化部署。 前台商城系统包含首页门户、商品推荐、商品搜索、商品展示、购物车、订单流程、会员中心、客户服务、帮助中心等模块。 后台管理系统包含商品管理、订单管理、会员管理、促销管理、运营管理、内容管理、统计报表、财务管理、权限管理、设置等模块。
Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed
Declarative Continuous Deployment for Kubernetes
Incremental engine for long horizon agents 🌟 Star if you like it!
SimpleX - the first messaging network operating without user identifiers of any kind - 100% private by design! iOS, Android and desktop apps 📱!
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
🪓 An orchestration engine for background tasks, AI agents, and durable workflows
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
High-Performance server for NATS.io, the cloud and edge native messaging system.
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Apache Doris is a real-time analytics and hybrid search database for AI agents.
All-in-one IoT Platform - Device management, data collection, processing and visualization.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
A high-performance observability data pipeline.
httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.
An orchestration platform for the development, production, and observation of data assets.
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
Restate is the platform for building resilient applications that tolerate all infrastructure faults w/o the need for a PhD.
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
A modern replacement for Redis and Memcached
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
The polyglot queue: Fast and reliable background jobs in Go, Java, JS, Ruby, and Rust on Postgres/SQLite.
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
BullMQ - Message Queue and Batch processing for NodeJS, Python, .NET, Elixir, Rust and PHP based on Redis or PostgreSQL
The leading workflow orchestration platform. Run stateful step functions and AI workflows on serverless, servers, or the edge.
Business intelligence as code: build fast, interactive data visualizations in SQL and markdown
Queueing jobs in Postgres from Node.js like a boss
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Python SQL Parser and Transpiler
😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
QuestDB is a high performance, open-source, time-series database
Open Source Feature Flags, Experimentation, and Product Analytics
Eclipse Mosquitto - An open source MQTT broker
The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
Cloud-native OSS search engine for observability
最新Docker容器技术,从真实案例中学习最佳实践!| Learn and understand Docker&Container technologies, with real DevOps practice!
IM Chat OpenClaw
Full Modular Monolith application with Domain-Driven Design approach.
The Data Engineering Cookbook
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Workflow Engine for Kubernetes
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
Always know what to expect from your data.
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
Upserts, Deletes And Incremental Processing on Big Data.
A distributed, fast open-source graph database featuring horizontal scalability and high availability
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
A curated list of awesome big data frameworks, ressources and other awesomeness.
Dapr is a portable runtime for building distributed applications across cloud and edge, combining event-driven architecture with workflow orchestration.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
mall-swarm是一套微服务商城系统,采用了 Spring Cloud Alibaba、Spring Boot 3.5、Sa-Token、MyBatis、Elasticsearch、Docker、Kubernetes等核心技术,同时提供了基于Vue的管理后台方便快速搭建系统。mall-swarm在电商业务的基础集成了注册中心、配置中心、监控中心、网关等系统功能。
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Open source RabbitMQ: core server and tier 1 (built-in) plugins
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
Agentic BI. Analytics at the speed of code ⚡️
Connect APIs, remarkably fast. Free for developers.
High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
The open source and enterprise-ready webhooks service 🦀
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Efficient reliable UDP unicast, UDP multicast, and IPC message transport
The most widely used Python to C compiler
A curated list of Domain-Driven Design (DDD), Command Query Responsibility Segregation (CQRS), Event Sourcing, and Event Storming resources
The Open Source Feature Store for AI/ML
Arkime is an open source, large scale, full packet capturing, indexing, and database system.
大数据入门指南 :star:
ZeroMQ core engine in C++, implements ZMTP/3.1