👾 Fast and simple video download library and CLI tool written in Go
Repository Index
55,640 visible repositories in the index
1–16 of 16 · Most stars
Elegant Scraper and Crawler Framework for Golang
Weighs the soul of incoming HTTP requests to stop AI crawlers
- defense
- security
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Pholcus is a distributed high-concurrency crawler software written in pure golang
- spider
- crowler
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Declarative data automation language and Go runtime for structured extraction workflows.
- golang
- query-language
- dsl
- go
A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
- bittorrent
- dht
- prowlarr
- radarr
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
teler-waf is a Go HTTP middleware that protects local web services from OWASP Top 10 threats, known vulnerabilities, malicious actors, botnets, unwanted crawlers, and brute force attacks.
- ids
- teler
- waf
- teler-ids
The unix-way web crawler
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Confluence Cloud to Markdown crawler with incremental updates and local link rewriting