TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Repository Index
63,269 visible repositories in the index
1–14 of 14 · Most stars
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
Pholcus is a distributed high-concurrency crawler software written in pure golang
- spider
- crowler
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Modern, easy to use download automation for torrents and usenet.
- torrent
- bittorrent
- irc
- announce
Translated🌌 Your Self-hosted Comment System.
- comment-system
- blog
- self-hosted
- golang
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
Execution engine powering Flowbaker, a no code platform to build, connect, and automate workflows.
- agents
- ai
- aiagents
- automation
RobustIRC - an IRC network without netsplits, implemented in Go using the Raft consensus algorithm
- go
- golang
- irc
- raft
Open-source AI alternative to Buffer and Opus Clip: Claude Code makes on-brand videos, posts them to 9 platforms and drafts replies. Self-hosted, your own keys.
Seamlessly fetch paginated data from any source. Simple and high performance API scraping included!
- api-scraper
- indexer
- pagination
- paginator
A lightweight webhook action engine — execute custom commands, scripts, and forward webhooks based on YAML rules when webhook requests arrive.
Lightweight Go SDK for Yandex Messenger Bot API — type-safe client with retry, rate-limit handling, and service-oriented architecture
- api-client
- bot
- bot-api
- chatbot