Repository Index

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

• webcrawler
• scrapy
• crawlab
• spiders-management

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

• go
• golang
• goquery
• colly