A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
Repository Index
59,621 visible repositories in the index
1–24 of 39 · Most stars
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
- golang
- chrome
- chrome-debugging-protocol
- testing
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
TranslatedWeekly Go Online Meetup via Bilibili｜Go Night Reading｜Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
- golang
- gopher
- shenzhen
- offline
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
- mattermost
- xmpp
- telegram
- gitter
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
A lightweight, secure, and feature-rich Discord terminal (TUI) client.
- go
- discord
- golang
- discordo
TranslatedGolang WeChat SDK
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
🔍 gowitness - a golang, web screenshot utility using Chrome Headless
- chrome
- headless
- screenshot
- security
A dead simple Go library for sending notifications to various messaging services.
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
- endpoints
- endpoint-discovery
- bugbounty
- crawler
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
TranslatedXianyu (Idle Fish) multi-account automatic shipping and message reply system, built with Go and React
- xianyu
- xianyu-assistant
- xianyu-auto-reply
- xianyu-bot
Translated🌌 Your Self-hosted Comment System.
- comment-system
- blog
- self-hosted
- golang
Vocat is an open-source web control panel and engineering toolkit for Quectel EC20/EC25-class cellular modems. It combines modem discovery, live radio status, AT and USSD terminals, SMS, WiFi Calling, eSIM management, network selection, proxy routing, notifications, audit logs, and release automation in one self-contained service.
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
Steam's protocol in Go to allow automation of different actions on the Steam network without running an actual Steam client. Includes APIs for friends, chatting, trading, trade offers and TF2 crafting.
- steam
- steamkit
Run a web page like a program
- cross-platform
- golang
- headless-browser
- purego