TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Repository Index
62,771 visible repositories in the index
1–3 of 3 · Most stars
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
- scrapy
- distributed
- webspider
- scrapyd
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:
- scrapy
- scrapyd
- scrapyd-ui
- scrapyd-api