Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
104,105 visible repositories in the index
1–17 of 17 · Most stars
Turn any website into a structured API. Extract, automate, search and monitor the web.
⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
TranslatedA WeChat official account crawler interface based on Sogou WeChat search
HTTrack Website Copier, copy websites to your computer (Official repository)
job-ops: DevOps principles applied to job hunting. A self-hosted pipeline to track, analyze, and assist your application process
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis
Library to scrape and clean web pages to create massive datasets.
Real time google image scraper. The process is automated by sending HTTP requests to retrieve image data which is then parsed and saved.
简单易用的Python爬虫框架,QQ交流群:597510560
Google, Naver multiprocess image web crawler (Selenium)
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Code to download, process, and analyse the Online Encyclopedia of Integer Sequences