🕵️♂️ Collect a dossier on a person by username from 6K websites
- osint
- social-network
- identification
- socmint
100,822 visible repositories in the index
1–24 of 102 · Most stars
🕵️♂️ Collect a dossier on a person by username from 6K websites
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
Elegant Scraper and Crawler Framework for Golang
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Turn any website into a structured API. Extract, automate, search and monitor the web.
A next-generation crawling and spidering framework.
TranslatedDouyin / TikTok platform tool for downloading works and collecting data
⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
TranslatedXiaohongshu (XiaoHongShu, RedNote) link extraction / post collection tool
Fast, reliable browser automation for web crawling, testing, scraping, and stealth. Bypasses CAPTCHAs with CDP Mode. Integrations include pytest, Playwright, MCP, and CLI.
TranslatedAn online tool for batch downloading WeChat Official Account articles. It supports exporting read count and comment data, requires no environment setup, and can be used through an online website. It supports private deployment with docker and deployment on Cloudflare. It supports downloading various file formats, and the HTML format can restore article layout and styles 100%.
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
TranslatedA batch download tool for WeChat Official Account articles, supporting download of comments and collections, supporting saving html/mhtml/md/pdf/docx/csv files, saving images, videos, and audio files within articles, and supporting MCP/Skill invocation
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
TranslatedPython API for JMComic | Provides a Python API for accessing 禁漫天堂 (JMComic), supporting both the web and mobile clients | 禁漫天堂 (JMComic) GitHub Actions downloader🚀
The AI local media metadata crawler
A Chrome DevTools Protocol driver for web automation and scraping.
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
TranslatedA WeChat official account crawler interface based on Sogou WeChat search
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.