Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
Repository Index
100,822 visible repositories in the index
1–24 of 36 · Most stars
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
TranslatedGet the real streaming URLs (live stream sources) and danmaku (live comments) of 58 live streaming platforms, including Douyu, Huya, Bilibili, Douyin, and Kuaishou. The live stream sources can be played in players such as PotPlayer and flv.js.
- douyu
- huya
- bilibili
- python
The next web scraper. See through the <html> noise.
TranslatedHands-on 🐍 web crawlers for many kinds of websites and e-commerce data 🕷. Includes 🕸: Taobao products, WeChat official accounts, Dianping, Qichacha, job recruitment sites, Xianyu, Alibaba tasks, Cnblogs, Weibo, Baidu Tieba, Douban movies, Baotu (ibaotu.com), Quanjing (quanjing.com), Douban music, a provincial drug administration, Sohu News, machine learning text collection, FOFA asset collection, Autohome, National Bureau of Statistics, Baidu keyword indexed-page counts, spider pool directories, Toutiao (Today's Headlines), Douban movie reviews, Ctrip, Xiaomi App Store, Anjuke, Tujia homestays ❤️❤️❤️. WeChat crawler showcase project:
- python3
- crawler
- baidu-tieba
- taobao-spider
TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.
Extract any website's complete design system with one command. DTCG tokens, semantic+primitive+composite, MCP server for Claude Code/Cursor/Windsurf, multi-platform emitters (iOS SwiftUI, Android Compose, Flutter, WordPress), Tailwind v4, Figma variables, shadcn/ui, CSS health audit, WCAG remediation, Chrome extension. MIT, Playwright, Node 20+.
All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers
- scanner
- crawler
- information-gathering
- admin-scanner
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos, contact information, followers, following and much more.
- facebook-page-insights
- instagram-followers-scraper
- facebook-ads-metrics
- facebook-contact-info
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
- scraper
- twitter-api
- twitter-bot
Export tweets, bookmarks, lists and much more from Twitter(X) web app. (推文/书签/收藏/列表导出工具)
- archiver
- exporter
- twitter-api
Generate and download e-books from online sources.
- lightnovel
- termux
- web-scraper
- console-app
Fast website link checker in Go
Point it at any web page and it finds the video, transcodes it, and streams it to your TV in real time. It even burns in subtitles.
多播放源自动采集在线影视网站 golang vue
- axios
- elementplus
- film
- gin
Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.
Multi-engine MCP server, CLI, and local daemon for agent web search and content retrieval — skill-guided workflows, no API keys.
Flexible Node.js crawler library
Retrieve author and publication information from Google Scholar in a friendly, Pythonic way without having to worry about CAPTCHAs!
- scholar
- googlescholar
- scholarly-articles
- scholarly-communications
简单易用的Python爬虫框架,QQ交流群:597510560
ZonyLrcToolsX 是一个能够方便地下载歌词的小软件。
- lyrics-fetcher
- taglib
- mp3
- dotnet-core