🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!
- website-monitor
- website-monitoring
- change-detection
- monitoring
Pythonic HTML Parsing for Humans™
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
新闻网页正文通用抽取器 Beta 版.
- python3
- webcrawler
- webspider
Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis
Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.
💬 RasaGPT is the first headless LLM chatbot platform built on top of Rasa and Langchain. Built w/ Rasa, FastAPI, Langchain, LlamaIndex, SQLModel, pgvector, ngrok, telegram
Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.
vulnx 🕷️ an intelligent Bot, Shell can achieve automatic injection, and help researchers detect security vulnerabilities CMS system. It can perform a quick CMS security detection, information collection (including sub-domain name, ip address, country information, organizational information and time zone, etc.) and vulnerability scanning.
- crawler
- information-gathering
- cms-detector
- cloudflare-detection
The repository host gives no description.
Find web directories without bruteforce
The repository host gives no description.
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
Scrapy project to scrape public web directories (educational) [DEPRECATED]
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people and characters
- imdb
- movies
- actors
- cinema
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
- articles
- articles-data
- crawler
- datasets-preparation
Automated lending on Cryptocurrency exchanges Poloniex and Bitfinex
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
ARES - authorized red-team engagement automation with dashboard, campaign scope, module orchestration, OPSEC controls, encrypted vault, and reporting.
- active-directory
- cloud-security
- defensive-validation
- fastapi
660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.
Z-Hound — a CSV-first collector that finds public threat reports and extracts structured IoCs without a database, LLM, or API key.
Python/FastAPI application for monitoring Brazilian public-sector IT opportunities, with automated data collection, job filters, and official-source tracking.