Repository Index

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

• webcrawler
• scrapy
• crawlab
• spiders-management

Real-time monitor and web admin for Celery distributed task queue

• celery
• task-queue
• monitoring
• administration

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

• crawler
• dotnetcore
• cross-platform
• csharp

Fast and distributed workflow runner using ActiveJob and Redis

• workflows
• ruby
• sidekiq
• graph

"Security Incidents In A Box!" A modular, menu-driven, cross-platform tool for building customized, time-delayed, distributed security events. Easily create custom event chains for Blue- & Red Team drills and sensor / alert mapping. Red Teams can create decoy incidents, distractions, and lures to support and scale their operations. Build event sequences ("narratives") to simulate realistic scenarios and generate corresponding network and filesystem artifacts.

• pentesting
• hacking
• hacking-tool
• pentest-tool

A scalable, mature and versatile web crawler based on Apache Storm

• web-crawler
• apache-storm
• distributed
• java

RedisSMQ: A high-performance Redis message queue for Node.js

• redis
• background-jobs
• brokers
• dead-letter-queue

iWF is a Workflow-As-Code microservice orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes

• golang
• workflow
• distributed-system
• orchestration

Easy C# API for Distributed Background Tasks/Jobs for .NET Core.

• netcore
• csharp
• aspnet
• netstandard

Maze Applied Reinforcement Learning Framework

• reinforcement-learning
• python
• machine-learning
• simulation

The type-safe, event-driven workflow orchestration library that scales with your business. Build robust, distributed workflows in Go with compile-time safety, automatic retries, and horizontal scaling out of the box. Integrate with your existing tech stack using adapters 🚀

• eventdrivenarchitecture
• golang
• kafka
• reflex

state machine which runs code

• golang
• workflows
• state-machine
• actor-model

Ratus is a RESTful asynchronous task queue server. It translated concepts of distributed task queues into a set of resources that conform to REST principles and provides a consistent HTTP API for various backends.

• distributed-systems
• priority-queue
• task-queue
• task-scheduler

Distributed batch data processing framework

• cassandra
• rabbitmq
• go
• batch-processing

Durable execution framework for AI agents in Go. Keeps agent state, tool calls, and execution loops resilient across process crashes.

• agentic-ai
• ai
• ai-agents
• golang

Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.

• ruby
• rake
• pwrake
• parallel

Distributed Automation for XNAT

• xnat
• python
• dax

Distributed, Scalable, Fault-Tolerant, Minimalistic Workflow Engine - No DAGs, No YAML, No Cumbersome Diagrams, Just Code

• business-process
• kafka
• kafka-streams
• micorservices

Go-native distributed job scheduler with delayed tasks, batched Redis coordination, retries, lease-based recovery, and versioned queue partitioning.

• distributed
• distributed-scheduler
• golang
• scheduler