👾 Fast and simple video download library and CLI tool written in Go
Repository Index
142,187 visible repositories in the index
1–24 of 40 · Most stars
Elegant Scraper and Crawler Framework for Golang
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Declarative data automation language and Go runtime for structured extraction workflows.
- golang
- query-language
- dsl
- go
Swiss-army tool for scraping and extracting data from online assets, made for hackers
- css
- curl
- gjson
- json
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
- endpoints
- endpoint-discovery
- bugbounty
- crawler
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
- golang
- go
- cloning
- crawler
Web Scraper in Go, similar to BeautifulSoup
- golang
- go
- webscraper
- webscraping
Self-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a real browser: ranked results and clean page Markdown. API, CLI, MCP 🎉
- baidu
- serp
- serpapi
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
Go HTTP client with browser-identical TLS/HTTP2 fingerprinting. Bypass bot detection by perfectly mimicking Chrome, Firefox, and Safari at the cryptographic level (JA3/JA4, Akamai fingerprint, header order). Supports HTTP/1.1, HTTP/2, HTTP/3, sessions, cookies, and proxies.
- anti-bot
- bot-detection
- browser-fingerprint
- browser-fingerprinting
Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.
开箱即用的本地 JAV/视频 刮削、管理、播放软件,支持命令行一键安装和 Docker 部署。只需简单添加目录,即可打造你的私人 JAV/视频 媒体库,带给你顶级的浏览体验,懒人必备。| Your local JAV/video manager.
- jav
- porn
- javbus
- javlibrary
MetaTube SDK & API Server in Golang
- gin
- gorm
- pigo-face-detection
- postgresql
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.
- web-scraping
- man-in-the-middle
- proxy-server
- mitm
A high-performance proxy rotation engine with automated IP management and real-time health monitoring
- http
- http-proxy
- ip-rotation
- proxy
Fast, stateless CLI for web search and scrape. Built for AI agents.
A prometheus exporter which scrapes remote JSON by JSONPath