Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
132,241 visible repositories in the index
1–24 of 43 · Most stars
Scrapy, a fast high-level web crawling & scraping framework for Python.
🕵️♂️ Collect a dossier on a person by username from 6K websites
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A next-generation crawling and spidering framework.
API Support for your favorite torrent trackers
The RSS feed for websites missing it
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
The Unofficial TikTok API Wrapper In Python
Dark Web OSINT Tool
An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices (e.g., jobs, scholarships); and sends a structured summary with a calculated match score directly to the assigned messenger number.
Downloads albums from the web in bulk for archive purposes
Google Maps Scraper & Lead Generation Tool. Extract 50+ data points including business emails, phone numbers, and social profiles. Includes enrichment features, API access, and a generous free plan with 200 searches/month.
Scrape Facebook public pages without an API key
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
A browser testing and web crawling library for PHP and Symfony
HTML parsing and querying with CSS selectors
Claim available free game promotions from the Epic Games Store.
⛏⚽ Scrape soccer data from Club Elo, ESPN, FBref, Football-Data.co.uk, Sofascore, SoFIFA, Understat and WhoScored.
Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App
Financial Data Extraction from Investing.com with Python
Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess
Simple random DNS, HTTP/S internet traffic noise generator
Download website to local directory (including all css, images, js, etc.)
Pull current and historical baseball statistics using Python (Statcast, Baseball Reference, FanGraphs)