Ranked by Attention Score: stars gained over the last 180 complete days
32 repositories by Attention Score · May 8, 2026 – Nov 3, 2026 (UTC)
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Python scraper based on AI
TranslatedA multi-task real-time/scheduled monitoring and intelligent analysis system for Xianyu (闲鱼), built with Playwright and AI, and equipped with a full-featured back-end management UI. It helps users find the products they like among the vast number of items on Xianyu.
TranslatedAn online tool for batch downloading WeChat Official Account articles. It supports exporting read count and comment data, requires no environment setup, and can be used through an online website. It supports private deployment with docker and deployment on Cloudflare. It supports downloading various file formats, and the HTML format can restore article layout and styles 100%.
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
Download pictures (or videos) along with their captions and other metadata from Instagram.
A next-generation crawling and spidering framework.
📊 APIs for Web Scraping, Browser Automation, and E2E Testing. CDP Mode & Stealthy Playwright Mode bypass bot-detection/CAPTCHAs.
Python ProxyPool for web spider
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A Chrome DevTools Protocol driver for web automation and scraping.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
TranslatedA new-generation web crawler platform that defines crawler workflows graphically, so you can build a crawler without writing code.
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)
Pythonic HTML Parsing for Humans™
Find web directories without bruteforce
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
A browser testing and web crawling library for PHP and Symfony
The next web scraper. See through the <html> noise.
Mechanize is a ruby library that makes automated web interaction easy.
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of User Interface or API, also known as Headless browser. It can be used to run automated tests / snapshots and manipulate websites using Javascript.
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.