Repository Index

TranslatedSome interesting examples of python crawlers that are friendly to beginners. Mainly crawls websites such as 淘宝 (Taobao, "searching for treasure"), 天猫 (Tmall, "sky cat"), 微信 (WeChat, "micro message"), 微信读书 (WeChat Reading), 豆瓣 (Douban, "bean petal"), and QQ.

• crawler
• spider
• taobao
• tmall

TranslatedPython scripts. Simulated login to Zhihu, web crawlers, working with Excel, WeChat official accounts, remote power-on

• python
• crawler
• wechat
• excel

TranslatedPython implementations of machine learning algorithms

List of libraries, tools and APIs for web scraping and data processing.

• web-scraping
• captcha-bypass
• captcha-recaptcha
• crawling

A collection of awesome web crawler,spider in different languages

• web-crawler
• crawler
• web-scraper
• spider

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?

• bot
• detection
• stealth
• scraper

TranslatedSome interesting Python crawlers and data analysis projects

TranslatedWeb crawler basics, intermediate web crawling, advanced web crawlers

TranslatedCollection of China illegal cases about web crawler. This project compiles news, materials, and laws and regulations related to lawsuits and violations involving web crawler developers in mainland China. It aims to help web crawler industry practitioners working in mainland China understand China's relevant laws and avoid crossing the red lines of data compliance.

• china
• law
• crawler

TranslatedPython web crawler tutorial that takes you from zero to one, covering JS reverse engineering, Selenium, Tesseract OCR recognition, the use of MongoDB, and the Scrapy framework

Scrape Google AI Mode responses without blocks on a large scale.

• ai-mode
• google-ai
• google-ai-mode
• proxy-scrape

A curated list of awesome ETL frameworks, libraries, and software.

• awesome
• awesome-list
• etl

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

• scrapy
• python
• crawl
• crawler

Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.

• google
• google-trend
• google-trend-api
• google-trend-strategy

A curated list of awesome puppeteer resources.

• puppeteer
• headless-chrome
• awesome
• awesome-list

Leaked GPTs Prompts Bypass the 25 message limit or to try out GPTs without a Plus subscription.

• ai
• awesome
• awesome-list
• gpts

Flutter UI challenge (with Box2D physic)- Smart washing machine app

• ui
• challenge
• ui-challenge
• flutter-apps

:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

• aws
• textract
• lambda
• dynamodb

Various Jupyter notebooks about Common Crawl data

• jupyter-notebook
• common-crawl
• aws-athena
• webarchiving