A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
Repository Index
47,133 repositories indexed
1–22 of 22 · Most stars
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
- chat
- chat-application
- chatapp
- go
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
webhook is a lightweight incoming webhook server to run shell commands
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
Fast website link checker in Go
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
The unix-way web crawler
Run a web page like a program
- cross-platform
- golang
- headless-browser
- purego
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
- readability
- scraper
- opengraph
Go package for scraping website recipes
- go
- golang
- recipe
- recipes
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
- browser-automation
- cdp
- chrome-devtools-protocol
- golang
A toolkit makes it easier to archive webpages to IPFS
- archiving
- webpage
- ipfs
A lightweight webhook action engine — execute custom commands, scripts, and forward webhooks based on YAML rules when webhook requests arrive.
Go http.Client with a byte-exact browser TLS fingerprint (uTLS) — survive JA3/JA4 blocking without a headless browser.
- akamai-fingerprint
- anti-bot
- cloudflare-bypass
- go
Open-source real-time Voice AI infrastructure in Go. Stream audio via WebRTC or WebSocket, connect STT → LLM → TTS pipelines, and build scalable voice agents and conversational AI applications.
- audio-streaming
- conversational-ai
- generative-ai
- llm