A scalable, mature and versatile web crawler based on Apache Storm
- web-crawler
- apache-storm
- distributed
- java
111,085 visible repositories in the index
721–744 of 1,112 · Most stars
A scalable, mature and versatile web crawler based on Apache Storm
Dapr user documentation, used to build docs.dapr.io
SiteOne Crawler is a cross-platform website crawler and analyzer for SEO, security, accessibility, and performance optimization—ideal for developers, DevOps, QA engineers, and consultants. Supports Windows, macOS, and Linux (x64 and arm64).
📚 Real-world OpenClaw automation examples from Moltbook
A tool for pixiv.net. 人人可用的P站爬虫
A simple Web based BOT for WhatsApp™ in NodeJS 😜. Working as of 📅 Feb 4th, 2024
构建受监督的、自我进化的 Agent 组织的基础设施 | Infrastructure for supervised, self-improving agent organization. 飞书/Telegram 手机端运行 Claude Code 或 Kimi Code(双引擎,两家原生订阅直接用),共享记忆、Agent 工厂、定时任务、通信总线。
Fetch X/Twitter tweets, replies, timelines, search, Lists and X Articles without login or API keys. Python CLI + library for AI agents: FxTwitter / Nitter / browser backends with auto fallback, unified JSON, optional SQLite archive. 免登录免 API 抓取推特推文。
Build browser agents for real world tasks
扫描“微信读书”已购图书并下载本地PDF的爬虫
Reconmap is a collaboration-first security operations platform for infosec teams and MSSPs, enabling end‑to‑end engagement management, from reconnaissance through execution and reporting. With built-in command automation, output parsing, and AI‑assisted summaries, it delivers faster, more structured, and high‑quality security assessments.
A Blazing fast Security Auditing tool for Kubernetes
Temporal Go SDK
The Learning Hub for UoL's Online CS Students
A task based API for taking screenshots and scraping text from websites.
A configurable web spider with a easy-to-use web console
Apache Block Bad Bots, (Referer) Spam Referrer Blocker, Vulnerability Scanners, Malware, Adware, Ransomware, Malicious Sites, Wordpress Theme Detectors and Fail2Ban Jail for Repeat Offenders
This app can now use Android, just like a human.
OSINT Project. Collect information from a mail. Gather. Profile. Timeline.
User-defined explicit automation for Android
Next generation of Twidere for Android
SpiderSuite (web security crawler) releases, wiki and roadmap
🥝 Next generation of the Kiwi IRC web client
Universal content reader MCP Server for 10+ platforms