🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
🧡 Everything is RSSible
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
Elegant Scraper and Crawler Framework for Golang
A little like that j-thing, only in Go.
- selector-strings
- jquery
- html-parsing
- goquery
Pythonic HTML Parsing for Humans™
📊 Blazing fast Python framework for web crawling, testing, scraping, and stealth. Supports pytest. Bypasses bot-detection with CDP Mode. Compatible with Playwright.
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
This is a python API which allows you to get the transcript/subtitles for a given YouTube video. It also works for automatically generated subtitles and it does not require an API key nor a headless browser, like other selenium based solutions do!
TranslatedINFO-SPIDER is a crawler toolbox 🧰 that brings many data sources together in one place, designed to help users get their own data back safely and quickly. The tool's code is open source and the process is transparent. Supported data sources include GitHub, QQ Mail, NetEase Mail, Alibaba Mail, Sina Mail, Hotmail, Outlook Mail, JD.com, Taobao, Alipay, China Mobile, China Unicom, China Telecom, Zhihu, Bilibili, NetEase Cloud Music, QQ friends, QQ groups, generating a Moments (WeChat friend circle) photo album, browser browsing history, 12306, cnblogs (博客园), CSDN blog, OSChina blog, Jianshu.
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)
- swift
- swiftsoup
- parse
- html-document
:zap: A distributed crawler for weibo, building with celery and requests.
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Utilities for downloading the EPG (Electronic Program Guide) for thousands of TV channels from hundreds of sources.
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
A browser testing and web crawling library for PHP and Symfony
AutoCLI is a Blazing fast, memory-safe command-line tool — Fetch information from any website with a single command. Covers Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and 55+ sites, with support for controlling Electron desktop apps, integrating local CLI tools (gh, docker, kubectl), now powered by AutoCLI.ai .
Gospider - Fast web spider written in Go
Generate and download e-books from online sources.
多播放源自动采集在线影视网站 golang vue
- axios
- elementplus
- film
- gin
HTML parsing and querying with CSS selectors
An HTML DOM parser. It allows you to manipulate HTML. Find tags on an HTML page with selectors just like jQuery.
HomeAssistant 国家电网电费查询