Repository Index

A next-generation crawling and spidering framework.

• crawler
• web-spider
• gocrawler
• spider-framework

Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots

• chat
• chat-application
• chatapp
• go

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

• webcrawler
• scrapy
• crawlab
• spiders-management

webhook is a lightweight incoming webhook server to run shell commands

• webhook
• go
• devops
• deploy

Yet another WebUI for Nginx

• nginx
• webui
• vue
• self-hosted

A Chrome DevTools Protocol driver for web automation and scraping.

• cdp
• chrome-headless
• chrome-devtools
• chrome-devtools-protocol

🔍 gowitness - a golang, web screenshot utility using Chrome Headless

• chrome
• headless
• screenshot
• security

Fast website link checker in Go

• linter
• website
• golang

An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.

• telegram
• archive
• wayback-machine
• save-the-internet

Extract structured data from web sites. Web sites scraping.

• golang
• golang-library
• extract-data
• chrome-fetcher

Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.

• accessibility-service
• adb-alternative
• ai-agents
• android

The unix-way web crawler

• cli
• crawler
• golang-application
• unix-way

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

• golang
• web-crawler
• crawling
• scraping

Run a web page like a program

• cross-platform
• golang
• headless-browser
• purego

A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Extract web archive data using Wayback Machine and Common Crawl

• golang
• crawler
• commoncrawl
• wayback-machine

Type-safe, persistent, embedded task queues and background job runner w/ SQLite. Web monitoring UI included.

• go
• golang
• job-queue
• job-scheduler

You had one job, or more then one, which can be done in steps

• tasks
• task-runner
• task-scheduler
• jobs

Scalable, stateless selenium, playwright, devtools, mpc hub for Kubernetes cluster

• selenium
• webdriver
• automation
• kubernetes

Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.

• ai
• claude
• content-extraction
• go

Cron for Docker — a job scheduler that runs commands in containers, on the host, or as swarm services, with a web UI, health endpoints and webhook notifications.

• container
• cron
• docker-container
• scheduler

Webpage summary extractor using Facebook Open Graph and arc90's readability

• readability
• scraper
• opengraph

This plugin implements Telegram WebApp Auth and Telegram Login Widget for the pocketbase

• pocketbase
• telegram
• telegram-bot
• pocketbase-plugins

Go package for scraping website recipes

• go
• golang
• recipe
• recipes