Elegant Scraper and Crawler Framework for Golang
Repository Index
77,852 visible repositories in the index
1–24 of 30 · Most stars
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
TranslatedAn API framework with a modular design based on Gin, which wraps commonly used features, is simple to use, and is dedicated to fast business development. For example, it supports cors cross-origin requests, jwt signature verification, zap log collection, panic exception capture, trace link tracing, prometheus monitoring metrics, swagger document generation, viper configuration file parsing, the gorm database component, the gormgen code generation tool, the graphql query language, errno unified definition of error codes, the use of gRPC, cron scheduled tasks, and so on.
- golang
- go
- api-gin
- gin
A lightweight, secure, and feature-rich Discord terminal (TUI) client.
- go
- discord
- golang
- discordo
Swiss-army tool for scraping and extracting data from online assets, made for hackers
- css
- curl
- gjson
- json
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
- endpoints
- endpoint-discovery
- bugbounty
- crawler
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
Private messaging system that hides metadata
Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
- golang
- go
- cloning
- crawler
Web Scraper in Go, similar to BeautifulSoup
- golang
- go
- webscraper
- webscraping
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
The unix-way web crawler
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
- golang
- unmarshall
- goquery
- selectors
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Disgo is the next generation of Discord API Consumption. Create a Discord Bot with Go using this Discord API Wrapper (SDK Client).
- discord
- discord-api
- discord-bot
- go
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
- readability
- scraper
- opengraph