Elegant Scraper and Crawler Framework for Golang
Repository Index
78,351 visible repositories in the index
1–24 of 25 · Most stars
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Swiss-army tool for scraping and extracting data from online assets, made for hackers
- css
- curl
- gjson
- json
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
- endpoints
- endpoint-discovery
- bugbounty
- crawler
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
- golang
- go
- cloning
- crawler
Web Scraper in Go, similar to BeautifulSoup
- golang
- go
- webscraper
- webscraping
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
The unix-way web crawler
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
- golang
- unmarshall
- goquery
- selectors
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
- readability
- scraper
- opengraph
Go package for scraping website recipes
- go
- golang
- recipe
- recipes
Text Generation using Markov Chains fed by 4chan APIs
- scrapper
- data-mining
- markov-chain
Seamlessly fetch paginated data from any source. Simple and high performance API scraping included!
- api-scraper
- indexer
- pagination
- paginator
pocketbase LLM docs scraper
- docs
- llm
- pocketbase
- scraper
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
- browser-automation
- cdp
- chrome-devtools-protocol
- golang