Repository Index

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

• crawler
• crawling
• crawling-python
• playwright

Scrapy, a fast high-level web crawling & scraping framework for Python.

• python
• scraping
• crawling
• framework

🕵️‍♂️ Collect a dossier on a person by username from 6K websites

• osint
• social-network
• identification
• socmint

Python scraper based on AI

• scraping
• scraping-python
• llm
• web-crawler

Pythonic HTML Parsing for Humans™

• html
• scraping
• python
• requests

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

• chromedriver
• selenium
• webdriver
• chrome

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

• web-scraping
• text-extraction
• nlp
• html2text

Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

• datadome
• perimeterx
• akamai
• antidetect-browser

Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.

• competitive-programming
• automation
• scraping
• atcoder

:scissors: High performance, multi-threaded image scraper

• python
• pypi
• scraper
• scraping