🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Repository Index
65,764 visible repositories in the index
1–13 of 13 · Most stars
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.
Malwoverview is a first response tool for threat hunting across VirusTotal, Hybrid Analysis, URLHaus, Polyswarm, Malshare, Alien Vault, Malpedia, Malware Bazaar, ThreatFox, Triage, IPInfo, Shodan, AbuseIPDB, GreyNoise, URLScan.io, Whois/RDAP, NIST, and VulnCheck. Supports LLM enrichment, IOC extraction, YARA scanning, and Android analysis.
- malware
- virustotal
- malpedia
- urlhaus
Use Google News Scraper API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.
- google-news-scraper
- google-scraper
- google-scraper-api
- news-scraper
Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing
- parameter-finder
- parameter
- bugbounty
- content-discovery
JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.
🌐 The easiest way to parse and modify URLs in Python.
- python
- python3
- url
- urls
Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal, GhostArchive & Intelligence X!
a small library for extracting rich content from urls
- oembed
- python
Make AI output sound human. Strips AI-isms (sycophancy, stock vocab, hedging stacks, em-dash pileups), preserves code/URLs/headings. Plugin for Claude Code, Cursor, Windsurf, Codex, Cline, Copilot, Gemini.
- ai-detection
- ai-plugin
- anti-slop
- claude-code
ChatGPT for MicroPython
Fetch and normalize parameterized URLs from the Wayback CDX API (OSINT, inspired by ParamSpider).
- bugbounty
- cdx
- cli
- kali-linux
A command-line tool and Python module to assign a property from one set of GeoJSON features to another