Ranked by Attention Score: stars gained over the last 30 complete days
32 repositories by Attention Score · Oct 5, 2026 – Nov 3, 2026 (UTC)
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Turn any website into an API your agent can call. Learns from the site's own frontend, heals itself when the site changes.
TranslatedA multi-task real-time/scheduled monitoring and intelligent analysis system for Xianyu (闲鱼), built with Playwright and AI, and equipped with a full-featured back-end management UI. It helps users find the products they like among the vast number of items on Xianyu.
Python scraper based on AI
Download pictures (or videos) along with their captions and other metadata from Instagram.
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
A next-generation crawling and spidering framework.
📊 APIs for Web Scraping, Browser Automation, and E2E Testing. CDP Mode & Stealthy Playwright Mode bypass bot-detection/CAPTCHAs.
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
TranslatedAn online tool for batch downloading WeChat Official Account articles. It supports exporting read count and comment data, requires no environment setup, and can be used through an online website. It supports private deployment with docker and deployment on Cloudflare. It supports downloading various file formats, and the HTML format can restore article layout and styles 100%.
Python ProxyPool for web spider
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A Chrome DevTools Protocol driver for web automation and scraping.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
TranslatedA new-generation web crawler platform that defines crawler workflows graphically, so you can build a crawler without writing code.
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Pythonic HTML Parsing for Humans™
Mechanize is a ruby library that makes automated web interaction easy.
A browser testing and web crawling library for PHP and Symfony
Find web directories without bruteforce
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
The next web scraper. See through the <html> noise.
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.