Elegant Scraper and Crawler Framework for Golang
Repository Index
47,133 repositories indexed
1–21 of 21 · Most stars
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
PufferPanel is an open source game server management panel, designed for both small networks and personal use
- game-server
- minecraft
- srcds
- panel
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Steam's protocol in Go to allow automation of different actions on the Steam network without running an actual Steam client. Includes APIs for friends, chatting, trading, trade offers and TF2 crafting.
- steam
- steamkit
The unix-way web crawler
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
- golang
- unmarshall
- goquery
- selectors
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
- readability
- scraper
- opengraph
Confluence Cloud to Markdown crawler with incremental updates and local link rewriting
Go package for scraping website recipes
- go
- golang
- recipe
- recipes
Text Generation using Markov Chains fed by 4chan APIs
- scrapper
- data-mining
- markov-chain
Seamlessly fetch paginated data from any source. Simple and high performance API scraping included!
- api-scraper
- indexer
- pagination
- paginator
pocketbase LLM docs scraper
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
- browser-automation
- cdp
- chrome-devtools-protocol
- golang
Go http.Client with a byte-exact browser TLS fingerprint (uTLS) — survive JA3/JA4 blocking without a headless browser.
- akamai-fingerprint
- anti-bot
- cloudflare-bypass
- go