🧡 Everything is RSSible
Repository Index
47,133 repositories indexed
1–24 of 32 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
TranslatedA multi-task real-time/scheduled monitoring and intelligent analysis system for Xianyu (闲鱼), built with Playwright and AI, and equipped with a full-featured back-end management UI. It helps users find the products they like among the vast number of items on Xianyu.
Download pictures (or videos) along with their captions and other metadata from Instagram.
- instagram-client
- instagram-feed
- downloader
Incredibly fast crawler designed for OSINT.
Browser Automation, End-to-End Testing, and Stealth in one library. Bypass bot-detection/CAPTCHAs with SeleniumBase CDP Mode.
- python
- selenium-python
- e2e-testing
- seleniumbase
TranslatedXiaohongshu (XiaoHongShu, RedNote) link extraction / post collection tool
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Dark Web OSINT Tool
- tor
- tor-network
- deepweb
- dark-web
Mechanize is a ruby library that makes automated web interaction easy.
- scraping
- web
- ruby
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Europe on platforms like ImmoScout24, Immowelt, eBay Kleinanzeigen and instantly delivers the results to you via Slack, Telegram, Email, Discord or ntfy, so you can focus on the more important things in life ;)
- fredy
- immowelt
- appartment
- appartments
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of User Interface or API, also known as Headless browser. It can be used to run automated tests / snapshots and manipulate websites using Javascript.
Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.
- agpl
- ai-agents
- anti-bot
- crawling
A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user
- automation
- ecommerce
- filamentphp
- laravel
A AI Browser run any site JavaScript from Python with the real Firefox engine. Headless SpiderMonkey + Necko for automation and scraping.
- anti-bot
- automation
- firefox
- headless-browser
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Get video info from Dailymotion, Vimeo, Wistia, and YouTube URLs.
Zotero extension that combats link rot by archiving webpages and journal articles.
Price tracker monitors of products and alerts you when prices drop. Supported tiki.vn, shopee, lotte.vn, ... Built with firebase https://pricetrack.web.app