Sign in

Repository Index

142,273 visible repositories in the index

Tags: Web scraper
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–24 of 33 · Most stars

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

  • pocket
  • wget
  • browser-bookmarks
  • pinboard

⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file

  • save-the-internet
  • procrastination
  • its-mine
  • no-more-404

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

  • endpoints
  • endpoint-discovery
  • bugbounty
  • crawler

Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

AIL framework - Analysis Information Leak framework. Project moved to https://github.com/ail-project

  • ail-framework
  • information-leak
  • information-security
  • analysis

🛡️ Автоматический сборщик MTProto‑прокси с уникальной фильтрацией «Умный комбайн» для автоматического сбора, анализа и фильтрации MTProto‑прокси. Отделяет **"золотые" прокси**, маскирующиеся под Yandex, VK, Gosuslugi и другие популярные RU‑сервисы (Fake‑TLS), от обычных.