Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
132,241 visible repositories in the index
1–16 of 16 · Most stars
Turn any website into a structured API. Extract, automate, search and monitor the web.
⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
job-ops: DevOps principles applied to job hunting. A self-hosted pipeline to track, analyze, and assist your application process
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis
Library to scrape and clean web pages to create massive datasets.
Real time google image scraper. The process is automated by sending HTTP requests to retrieve image data which is then parsed and saved.
简单易用的Python爬虫框架,QQ交流群:597510560
Google, Naver multiprocess image web crawler (Selenium)
A system of bots that collects clips automatically via custom made filters, lets you easily browse these clips, and puts them together into a compilation video ready to be uploaded straight to any social media platform. Full VPS support is provided, along with an accounts system so multiple users can use the bot at once. This bot is split up into three separate programs. The server. The client. The video generator. These programs perform different functions that when combined creates a very powerful system for auto generating compilation videos.
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Code to download, process, and analyse the Online Encyclopedia of Integer Sequences