TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Repository Index
78,351 visible repositories in the index
25–48 of 220 · Most stars
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
- ai-agent
- anti-bot
- antidetect-browser
- automation
Translated:orange_book: Chinese Xinhua Dictionary database. Includes xiehouyu (two-part allegorical sayings), idioms, words, and Chinese characters.
- data
- scraper
- chinese-traditional
- python3
A scalable web crawler framework for Java.
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.
- perimeterx
- nopecha
- ai-agents
- anti-captcha
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
- apify
- automation
- beautifulsoup
- crawler
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
- python
- machine-learning
- natural-language-processing
- web-mining
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
- python
- python-script
- image-download
- google-images
Lighter web automation with Python
- selenium
- selenium-python
- python
- python3
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
🔥 The fastest and powerful Python library for Instagram Private API 2026 with HikerAPI SaaS
- instagram-api
- instagram-client
- instagram-downloader
Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
- cdp
- chromium
- playwright
- puppeteer
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
Turn any webpage into structured data using LLMs
A Python module to bypass Cloudflare's anti-bot page.
- cloudflare
- cloudflare-bypass
- cloudflare-scrape
- anti-bot-page
The Unofficial TikTok API Wrapper In Python
- tiktok-api
- python
- api
- trending
TranslatedA WeChat official account crawler interface based on Sogou WeChat search
Official Bright Data CLI - scrape, search, and extract structured web data directly from your terminal.
💡 Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js
- offline-web-pages
- downloader
- scraper
- assets