Elegant Scraper and Crawler Framework for Golang
Repository Index
70,234 visible repositories in the index
1–24 of 38 · Most stars
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
- chat
- chat-application
- chatapp
- go
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
webhook is a lightweight incoming webhook server to run shell commands
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Swiss-army tool for scraping and extracting data from online assets, made for hackers
- css
- curl
- gjson
- json
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
- chrome
- headless
- screenshot
- security
A scalable social community platform powered by Gin backend and modern TypeScript/Vue frontend architecture
- forum
- go
- vue3
- bbs
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
- endpoints
- endpoint-discovery
- bugbounty
- crawler
Shadow any website for offline viewing, with the JavaScript stripped out
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
Translated🌌 Your Self-hosted Comment System.
- comment-system
- blog
- self-hosted
- golang
Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
- golang
- go
- cloning
- crawler
Web Scraper in Go, similar to BeautifulSoup
- golang
- go
- webscraper
- webscraping
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
Low-code platform
- go
- golang
- javascript
- low-code
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
The unix-way web crawler