🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Scrapy, a fast high-level web crawling & scraping framework for Python.
🕵️♂️ Collect a dossier on a person by username from 6K websites
- osint
- social-network
- identification
- socmint
Elegant Scraper and Crawler Framework for Golang
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
📊 Blazing fast Python framework for web crawling, testing, scraping, and stealth. Supports pytest. Bypasses bot-detection with CDP Mode. Compatible with Playwright.
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
A scalable web crawler framework for Java.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
The All in One Framework to Build Undefeatable Scrapers
- anti-bot
- anti-detection
- cloudflare-bypass
- cloudflare-scrape
Successor of Undetected-Chromedriver. Providing a blazing fast framework for web automation, webscraping, bots and any other creative ideas which are normally hindered by annoying anti bot systems like Captcha / CloudFlare / Imperva / hCaptcha
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
新闻网页正文通用抽取器 Beta 版.
- python3
- webcrawler
- webspider
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单,功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
🙌Easily download all the videos from TikTok(amemv).下载指定的 抖音(Douyin) 号的视频,抖音爬虫
Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.
Web Scraping Framework
Tool to help us buy hard to find items.
Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
Claim available free game promotions from the Epic Games Store.
- epicgames-freebies-claimer
- epicgames
- epic-games
- giveaway