Elegant Scraper and Crawler Framework for Golang
Repository Index
47,133 repositories indexed
1–8 of 8 · Most stars
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
The unix-way web crawler
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Confluence Cloud to Markdown crawler with incremental updates and local link rewriting