Sign in

Repository Index

100,822 visible repositories in the index

Tags: Web scraper
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–24 of 102 · Most stars

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!

  • website-monitor
  • website-monitoring
  • change-detection
  • monitoring

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

  • pocket
  • wget
  • browser-bookmarks
  • pinboard

TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.

⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file

  • save-the-internet
  • procrastination
  • its-mine
  • no-more-404

Fast, reliable browser automation for web crawling, testing, scraping, and stealth. Bypasses CAPTCHAs with CDP Mode. Integrations include pytest, Playwright, MCP, and CLI.

TranslatedAn online tool for batch downloading WeChat Official Account articles. It supports exporting read count and comment data, requires no environment setup, and can be used through an online website. It supports private deployment with docker and deployment on Cloudflare. It supports downloading various file formats, and the HTML format can restore article layout and styles 100%.

  • download
  • wechat
  • wechat-article
  • wechat-download

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

  • webcrawler
  • scrapy
  • crawlab
  • spiders-management

TranslatedA batch download tool for WeChat Official Account articles, supporting download of comments and collections, supporting saving html/mhtml/md/pdf/docx/csv files, saving images, videos, and audio files within articles, and supporting MCP/Skill invocation

A Chrome DevTools Protocol driver for web automation and scraping.

  • cdp
  • chrome-headless
  • chrome-devtools
  • chrome-devtools-protocol