A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
47,133 repositories indexed
1–15 of 15 · Most stars
A next-generation crawling and spidering framework.
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
TranslatedWeekly Go Online Meetup via Bilibili|Go Night Reading|Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
A Chrome DevTools Protocol driver for web automation and scraping.
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
Extract structured data from web sites. Web sites scraping.
Run a web page like a program
Ready to use Slack bot for lazy developers: start Jenkins jobs, watch Jira tickets, watch pull requests with AI support...
Translated🎮 Update profile README / pinned gist to contain your Steam playtime leaderboard.
Translated📊 Update profile README / pinned gist to contain your weekly WakaTime stats. This is a Golang implementation, see the original version https://github.com/matchai/waka-box. Display your weekly WakaTime language usage statistics in your profile README / pinned gist.
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
Go http.Client with a byte-exact browser TLS fingerprint (uTLS) — survive JA3/JA4 blocking without a headless browser.