78,351 visible repositories in the index
1–24 of 66 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
API Support for your favorite torrent trackers
⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
A scalable web crawler framework for Java.
A standalone version of the readability lib
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
Get the main content of any page as Markdown.
TranslatedSina Weibo crawler, uses Python to scrape Sina Weibo data
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
The All in One Framework to Build Undefeatable Scrapers
📜 Extract meaningful content from the chaos of a web page
:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis
Extract any website's complete design system with one command. DTCG tokens, semantic+primitive+composite, MCP server for Claude Code/Cursor/Windsurf, multi-platform emitters (iOS SwiftUI, Android Compose, Flutter, WordPress), Tailwind v4, Figma variables, shadcn/ui, CSS health audit, WCAG remediation, Chrome extension. MIT, Playwright, Node 20+.
TranslatedSina Weibo crawler: uses Python to crawl Sina Weibo data and download Weibo images and Weibo videos
Downloads albums from the web in bulk for archive purposes
🔮 A Node.js scraper for humans.
Automatically crawls proxy nodes on the public internet, de-duplicates and tests for usability and then provides a list of nodes
All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers
Abstraction layer over YouTube's internal API
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.