A complete and versatile web scraper.
Repository Index
78,850 visible repositories in the index
121–144 of 328 · Most stars
Scrape Google AI Mode responses without blocks on a large scale.
- ai-mode
- google-ai
- google-ai-mode
- proxy-scrape
There was no free Dictionary API on the web when I wanted one for my friend, so I created one.
Better GitHub statistics images for your profile, with stats from private repos too
- github-stats
- statistics
- profile
- visualizations
A Python module to bypass Cloudflare's anti-bot page.
- cloudflare
- anti-bot-page
- protected-page
- scrape
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
- scrapy
- distributed
- webspider
- scrapyd
freehire — the open-source search engine for job seekers
- ats
- careers
- hiring
- job-aggregator
Claim Free proxy list with United States IP addresses and use it for your projects.
- free-proxies
- free-proxies-for-web-scraping
- free-proxy
- free-proxy-ip
一个视频播放器，支持网盘直连、元数据刮削、IPTV、文件管理等 / A video player that supports direct connection to network disk, metadata scraping, IPTV, file management, etc.
- android-tv
- video-player
- iptv
- quark
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
- amazon-scraper
- amazon-scraping
- e-commerce-api
- web-scraping
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
- scrapy
- python
- crawl
- crawler
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:
- scrapy
- scrapyd
- scrapyd-ui
- scrapyd-api
Use Google News Scraper API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.
- google-news-scraper
- google-scraper
- google-scraper-api
- news-scraper
Easily collect responses from ChatGPT scraper by providing a prompt along with valid Web Scraper API credentials.
- chatgpt-api
- chatgpt-scraper
- llm-scraper
- chatgpt
Get account information, photos, videos, stories and comments.
- instagram-api
- instagram-php-scraper
- instagram-sdk
Stealth browser automation for AI agents over MCP and the Chrome DevTools Protocol: navigation, network hooks, DOM extraction, and pixel-accurate UI cloning.
Scrape Facebook public pages without an API key
- scraping
- facebook-scraping
- facebook-scraper
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
- php
- curl
- class
- api
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
新浪微博爬虫（Scrapy、Redis）
Free anti fingerprint browser, 指纹浏览器, 隐私浏览器, 防识别浏览器, 反识别浏览器, 防关联浏览器, 免费的web3空投专用指纹浏览器 https://virtualbrowser.cc/?src=github
- chromium-based
- multi-account
- privacy-tools
- proxy-support
An open source fingerprint browser based on Ungoogled Chromium. 指纹浏览器 隐私浏览器
- browser
- chromium
- fingerprint
- chrome