Pythonic HTML Parsing for Humans™
Repository Index
47,133 repositories indexed
25–48 of 89 · Most stars
Browser Automation, End-to-End Testing, and Stealth in one library. Bypass bot-detection/CAPTCHAs with SeleniumBase CDP Mode.
- python
- selenium-python
- e2e-testing
- seleniumbase
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
- ai-agent
- anti-bot
- antidetect-browser
- automation
A scalable web crawler framework for Java.
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.
- perimeterx
- nopecha
- ai-agents
- anti-captcha
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
- python
- machine-learning
- natural-language-processing
- web-mining
Lighter web automation with Python
- selenium
- selenium-python
- python
- python3
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
Mechanize is a ruby library that makes automated web interaction easy.
- scraping
- web
- ruby
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
A Python module to bypass Cloudflare's anti-bot page.
- cloudflare
- anti-bot-page
- protected-page
- scrape
A browser testing and web crawling library for PHP and Symfony
python parser for human readable dates
- hacktoberfest
A curated list of awesome puppeteer resources.
- puppeteer
- headless-chrome
- awesome
- awesome-list
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
- datadome
- perimeterx
- akamai
- antidetect-browser
Get info from any web service or page
- opengraph
- twitter-cards
- embeds
- scraping
SURF - Advanced Go HTTP client with Chrome/Firefox browser impersonation, HTTP/3 with QUIC fingerprinting, JA3/JA4 TLS emulation, and anti-bot bypass for web automation and scraping.
Async Python 3.6+ web scraping micro-framework based on asyncio
- asyncio
- aiohttp
- asyncio-spider
- crawler
❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Europe on platforms like ImmoScout24, Immowelt, eBay Kleinanzeigen and instantly delivers the results to you via Slack, Telegram, Email, Discord or ntfy, so you can focus on the more important things in life ;)
- fredy
- immowelt
- appartment
- appartments
A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
Simple web scraping for R