🧡 Everything is RSSible
Repository Index
90,854 visible repositories in the index
1–23 of 23 · Most stars
A scalable web crawler framework for Java.
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
- apify
- automation
- beautifulsoup
- crawler
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
- python
- python-script
- image-download
- google-images
💡 Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js
- offline-web-pages
- downloader
- scraper
- assets
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
- amazon
- amazon-scraper
- python
- web-scraper
基于 webmagic 的 Java 爬虫应用
- elasticsearch
- kibana
- zhihu
Download images from Pixiv and more!
Download images from Google, Bing, Baidu. 谷歌、百度、必应图片下载.
- image-downloader
- google-images
- bing
- baidu
Real time google image scraper. The process is automated by sending HTTP requests to retrieve image data which is then parsed and saved.
- google-image-scraper
- google-images-crawler
- image-scrape
- image-scraper
Find broken links, missing images, etc within your HTML.
Download website to local directory (including all css, images, js, etc.)
微博爬虫及配套工具箱,微博用户、话题、评论采集一网打尽。图片下载、情感分析,地理位置、关系网络、spammer 机器人识别等功能应有尽有。Docs:https://buyixiao.github.io/blog/weibo-super-spider.html 配套可视化网站:https://buyixiao.github.io/blog/one-stop-weibo-visualization.html
- weibo-spider
- weibocrawler
- weibo-comment-crawl
- emotion-analysis
Google, Naver multiprocess image web crawler (Selenium)
- crawler
- selenium
- deep-learning
- image-crawler
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
:scissors: High performance, multi-threaded image scraper
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
One-link downloader for the Mac (videos, documents, images, articles, web pages), with optional automatic YouTube monitoring that translates and subtitles new videos.
Extract URLs to stylesheets, scripts, links, images or HTML imports from HTML
Download Medium posts as clean Markdown, preserving structure, images, links, code blocks, and common embeds for plain Markdown or Jekyll workflows.
- ruby
- medium
- medium-articles
- markdown
pocketbase LLM docs scraper
- docs
- llm
- pocketbase
- scraper
Sitecore Content Hub Asset Importer. Allows importing image assets from various data sources into Sitecore Content Hub.