47,133 repositories indexed
1–24 of 25 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A little like that j-thing, only in Go.
CLI for common Playwright actions. Record and generate Playwright code, inspect selectors and take screenshots.
Pythonic HTML Parsing for Humans™
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)
Headless Chromium-based web performance metrics collector and monitoring tool
Find web directories without bruteforce
This is a repository for Microsoft Power Automate, Power Apps, and Azure Logic Apps connectors
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
htmlquery is golang XPath package for HTML query.
App-inspector is a mobile UI viewer in browser.
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
querySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer
Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!
Self-hosted, invisible CAPTCHA that detects AI agents driving real browsers. GDPR, WCAG 2.2 AA, and EAA: no cookies, no third-party data sharing, and no visual or audio puzzle. With a false-positive budget enforced in CI. Proof of work, drop-in for Turnstile and reCAPTCHA.
:vertical_traffic_light: Block malicious crawlers with Cloudflare Firewall Rules
TranslatedDeepSeek Blue Big Fat Fish Elimination Plan: a Chrome / Edge / Firefox browser extension that identifies and cleans up bot accounts (大肥鱼, Dafeiyu, "big fat fish") driven by the DeepSeek large model and AI agents in Bilibili comment sections. Offline rules + precise judgment by the DeepSeek API, AI comment detection, profile page identification, and one-click clearing by 鲸鱼娘 (Jingyuniang, "whale girl"). DeepSeek AI bot detector for Bilibili comments.
Python/FastAPI application for monitoring Brazilian public-sector IT opportunities, with automated data collection, job filters, and official-source tracking.
Local-first automation Agent & deterministic workflow engine — the secure control layer between AI and your data. Data stays local. Connect your own LLM / files / notes / databases via Connector API. ReAct reasoning · DAG/WAL/Saga/idempotent execution · MCP · offline-capable.
Capture, inspect, and replay webhooks with 90-day history + signature verification. Built on Cloudflare Workers + D1.