A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
47,133 repositories indexed
1–20 of 20 · Most stars
A next-generation crawling and spidering framework.
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
TranslatedWeekly Go Online Meetup via Bilibili|Go Night Reading|Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
webhook is a lightweight incoming webhook server to run shell commands
A Chrome DevTools Protocol driver for web automation and scraping.
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
Extract structured data from web sites. Web sites scraping.
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
Steam's protocol in Go to allow automation of different actions on the Steam network without running an actual Steam client. Includes APIs for friends, chatting, trading, trade offers and TF2 crafting.
Run a web page like a program
Ready to use Slack bot for lazy developers: start Jenkins jobs, watch Jira tickets, watch pull requests with AI support...
Translated🎮 Update profile README / pinned gist to contain your Steam playtime leaderboard.
Translated📊 Update profile README / pinned gist to contain your weekly WakaTime stats. This is a Golang implementation, see the original version https://github.com/matchai/waka-box. Display your weekly WakaTime language usage statistics in your profile README / pinned gist.
Private Telegram bot and API that downloads App Store IPAs with ipatool and decrypts them with UnfairPlay
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
Go-native browser automation via Chrome DevTools Protocol. No Node.js, no relay process, no external dependencies.
Go http.Client with a byte-exact browser TLS fingerprint (uTLS) — survive JA3/JA4 blocking without a headless browser.