Sign in

Repository Index

47,133 repositories indexed

Tags: Automation, bots, and scraping
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters (1)

Filters

Clear all filters

1–9 of 9 · Most stars

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

  • webcrawler
  • scrapy
  • crawlab
  • spiders-management

An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.

  • telegram
  • archive
  • wayback-machine
  • save-the-internet

Extract structured data from web sites. Web sites scraping.

  • golang
  • golang-library
  • extract-data
  • chrome-fetcher

A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

  • golang
  • unmarshall
  • goquery
  • selectors

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

  • go
  • golang
  • goquery
  • colly

Open-source real-time Voice AI infrastructure in Go. Stream audio via WebRTC or WebSocket, connect STT → LLM → TTS pipelines, and build scalable voice agents and conversational AI applications.