A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
140,194 visible repositories in the index
1–23 of 23 · Most stars
A next-generation crawling and spidering framework.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
GitHub Agentic Workflows
Swiss-army tool for scraping and extracting data from online assets, made for hackers
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
GoClaw - GoClaw is OpenClaw rebuilt in Go — with multi-tenant isolation, 5-layer security, and native concurrency. Deploy AI agent teams at scale without compromising on safety.
Polite, slim and concurrent web crawler.
Extract urls from text
小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
Slack Bot Framework
:paw_prints: Creeper - The Next Generation Crawler Framework (Go)
Extract structured data from web sites. Web sites scraping.
A developer experience centered on CUE. Unifies schemas, data models, deterministic and agentic code generation, workflow and task engine, dagger powered environments, coding assistant, and vscode extension; woven together on the CUE lattice. Squint harder if you can't see the cube :]
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Extract web archive data using Wayback Machine and Common Crawl
Disgo is the next generation of Discord API Consumption. Create a Discord Bot with Go using this Discord API Wrapper (SDK Client).
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
Text Generation using Markov Chains fed by 4chan APIs
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
Open-source real-time Voice AI infrastructure in Go. Stream audio via WebRTC or WebSocket, connect STT → LLM → TTS pipelines, and build scalable voice agents and conversational AI applications.