A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
55,140 visible repositories in the index
1–24 of 31 · Most stars
A next-generation crawling and spidering framework.
Bridge local AI coding agents (Claude Code, Cursor, Gemini CLI, Codex) to messaging platforms (Feishu/Lark, DingTalk, Slack, Telegram, Discord, LINE, WeChat Work). Chat with your AI dev assistant from anywhere — no public IP required for most platforms.
A little like that j-thing, only in Go.
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
Fast, scriptable CLI for the App Store Connect API. Automate TestFlight, builds, submissions, signing, analytics, screenshots, subscriptions, and more
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
The only Kubernetes dashboard you need
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
TranslatedXianyu (Idle Fish) multi-account automatic shipping and message reply system, built with Go and React
Translated🌌 Your Self-hosted Comment System.
Low-code platform
Vocat is an open-source web control panel and engineering toolkit for Quectel EC20/EC25-class cellular modems. It combines modem discovery, live radio status, AT and USSD terminals, SMS, WiFi Calling, eSIM management, network selection, proxy routing, notifications, audit logs, and release automation in one self-contained service.
htmlquery is golang XPath package for HTML query.
Extract structured data from web sites. Web sites scraping.
Steam's protocol in Go to allow automation of different actions on the Steam network without running an actual Steam client. Includes APIs for friends, chatting, trading, trade offers and TF2 crafting.
Framework for authoring workflows and activities running on top of the Cadence orchestration engine.
Run a web page like a program
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
Execution engine powering Flowbaker, a no code platform to build, connect, and automate workflows.
Extract web archive data using Wayback Machine and Common Crawl
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.
Webpage summary extractor using Facebook Open Graph and arc90's readability
Automatically import infrastructure to tfstate