🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
🕵️♂️ Collect a dossier on a person by username from 6K websites
- osint
- social-network
- identification
- socmint
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Crawl a site to generate knowledge files to create your own custom GPT from a URL
Download pictures (or videos) along with their captions and other metadata from Instagram.
- instagram
- instagram-client
- instagram-feed
- downloader
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
- python
- python-script
- image-download
- google-images
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
Official Bright Data CLI - scrape, search, and extract structured web data directly from your terminal.
Download an entire website from the Wayback Machine.
📜 Extract meaningful content from the chaos of a web page
Swiss-army tool for scraping and extracting data from online assets, made for hackers
Downloads albums from the web in bulk for archive purposes
- ripme
- java
- album
- downloader
Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language Models in this new AI era
Google Maps Scraper & Lead Generation Tool. Extract 50+ data points including business emails, phone numbers, and social profiles. Includes enrichment features, API access, and a generous free plan with 200 searches/month.
- google-maps-scraping
- google-maps
- google-maps-api
- google-maps-review
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
Utilities for downloading the EPG (Electronic Program Guide) for thousands of TV channels from hundreds of sources.
Turn any website into an API your agent can call. Learns from the site's own frontend, heals itself when the site changes.
Use Google News Scraper API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.
- google-news-scraper
- google-scraper
- google-scraper-api
- news-scraper
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos, contact information, followers, following and much more.
- facebook-page-insights
- instagram-followers-scraper
- facebook-ads-metrics
- facebook-contact-info
A metasearch library that aggregates results from diverse web search services
- python
- search
- metasearch
- ddgs
AutoCLI is a Blazing fast, memory-safe command-line tool — Fetch information from any website with a single command. Covers Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and 55+ sites, with support for controlling Electron desktop apps, integrating local CLI tools (gh, docker, kubectl), now powered by AutoCLI.ai .
Perplexity Scraper Track brand mentions, analyze rankings, and gain competitor intelligence from Perplexity. Get started in minutes.
- ai-scraper
- llm-api
- llm-scraper
- llm-scraping
Node.js scraper to get data from Google Play