TranslatedA WeChat official account crawler interface based on Sogou WeChat search
Repository Index
142,187 visible repositories in the index
73–96 of 627 · Most stars
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
- ai-agents
- automation
- claude-code
- claude-code-skills
💡 Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js
- offline-web-pages
- downloader
- scraper
- assets
Official Bright Data CLI - scrape, search, and extract structured web data directly from your terminal.
Declarative data automation language and Go runtime for structured extraction workflows.
- golang
- query-language
- dsl
- go
📈 Get real-time stocks from TradingView
- tradingview
- tradingview-api
- realtime-stocks
- stocks
The next web scraper. See through the <html> noise.
The All in One Framework to Build Undefeatable Scrapers
- anti-bot
- anti-detection
- cloudflare-bypass
- cloudflare-scrape
TranslatedHands-on 🐍 web crawlers for many kinds of websites and e-commerce data 🕷. Includes 🕸: Taobao products, WeChat official accounts, Dianping, Qichacha, job recruitment sites, Xianyu, Alibaba tasks, Cnblogs, Weibo, Baidu Tieba, Douban movies, Baotu (ibaotu.com), Quanjing (quanjing.com), Douban music, a provincial drug administration, Sohu News, machine learning text collection, FOFA asset collection, Autohome, National Bureau of Statistics, Baidu keyword indexed-page counts, spider pool directories, Toutiao (Today's Headlines), Douban movie reviews, Ctrip, Xiaomi App Store, Anjuke, Tujia homestays ❤️❤️❤️. WeChat crawler showcase project:
- python3
- crawler
- baidu-tieba
- taobao-spider
Redis-based components for Scrapy.
- scrapy
- crawler
- distributed
- redis
Distributed crawler powered by Headless Chrome
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis
- high-availability
- scrapy
- ipproxy
- distributed
A social networking service scraper in Python
- python
- scraper
- social-network
- social-media
TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.
Undetected version of the Playwright testing and automation library.
- automation
- bot
- bots
- botting
TranslatedAn AV metadata scraper that aggregates data from multiple sites
- python
- kodi
- emby
- jellyfin
Extract any website's complete design system with one command. DTCG tokens, semantic+primitive+composite, MCP server for Claude Code/Cursor/Windsurf, multi-platform emitters (iOS SwiftUI, Android Compose, Flutter, WordPress), Tailwind v4, Figma variables, shadcn/ui, CSS health audit, WCAG remediation, Chrome extension. MIT, Playwright, Node 20+.
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
Successor of Undetected-Chromedriver. Providing a blazing fast framework for web automation, webscraping, bots and any other creative ideas which are normally hindered by annoying anti bot systems like Captcha / CloudFlare / Imperva / hCaptcha
Jobs scraper library for LinkedIn, Indeed, ZipRecruiter, Glassdoor, SEEK, Naukri, HelloWork, Bayt & BDJobs
- indeed
- ziprecruiter
- linkedin-scraper
Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot
- python
- bot
- client
- python3
Swiss-army tool for scraping and extracting data from online assets, made for hackers
- css
- curl
- gjson
- json
YouTube video downloader in javascript.
- youtube
- video-downloader
- scraper
- youtube-downloader