The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
Repository Index
47,133 repositories indexed
1–11 of 11 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)
- swift
- swiftsoup
- parse
- html-document
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
- datadome
- perimeterx
- akamai
- antidetect-browser
Headless Chromium-based web performance metrics collector and monitoring tool
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
querySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer
- webcomponents
- shadow-dom
- javascript
- puppeteer
A simple yet highly configurable Mastodon bot that posts geotagged aerial imagery of a random location in the world.
This project focuses on automation of multi-level shadow root dom using java selenium. You can embed this plugin in your java selenium project.
- selenium
- selenium-webdriver
- selenium-java
- shadow-dom