A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
- amazon
- amazon-scraper
- api
- python
Repository Index
78,351 visible repositories in the index
169–192 of 277 · Most stars
An API for sending and receiving messages over web.whatsapp [Working as of 18th May 2018]
NewPipe's core library for extracting data from streaming sites
Specify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion
python ip proxy tool scrapy crawl. 抓取大量免费代理 ip，提取有效 ip 使用
- proxy
- python
- ipproxy
To extract article from given URL
- article-parser
- readability
- article-extractor
- content-extraction
A powerful no-code automation tool. Build workflows via drag-and-drop modules for web scraping, form filling and automated testing. | 一款功能强大的自动化工具，通过拖拽模块的方式快速构建自动化工作流，无需编写任何代码即可实现网页数据采集、表单填写、自动化测试等任务。【QQ交流群：115069513】
- ddddocr
- easyocr
- express-js
- face-recognition
Implementation of X/Twitter v1, v2, and GraphQL APIs
- api
- automation
- client
- scrape
A pure-python HTML screen-scraping library
Actively maintained, pure Python wrapper for the Twitter API. Supports both normal and streaming Twitter APIs.
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
- automation
- chromium
- screenshot
- web-scraping
OnionSearch is a script that scrapes urls on different .onion search engines.
- information-gathering
- osint
- osint-tools
- open-source-intelligence
Open Source Intelligence Interface for Deep Web Scraping
- ahmia
- deep-web-search
- osint
- tor
Finviz analysis python library.
- web-scraping
- fundamental-analysis
- stock-news
- technical-analysis
Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess
🕵️♀️ Validate schema, get schema change notifications, validate operations, find breaking changes, look for similar types, schema coverage
Download website to local directory (including all css, images, js, etc.)
The process of handling CAPTCHA when extracting public data from Amazon with Oxylabs Amazon Scraper API.
- amazon
- amazon-scraper
- amazon-scraper-api
- captcha
🏀 Python3 网络爬虫实战（部分含详细教程）猫眼 腾讯视频 豆瓣 研招网 微博 笔趣阁小说 百度热点 B站 CSDN 网易云阅读 阿里文学 百度股票 今日头条 微信公众号 网易云音乐 拉勾 有道 unsplash 实习僧 汽车之家 英雄联盟盒子 大众点评 链家 LPL赛程 台风 梦幻西游、阴阳师藏宝阁 天气 牛客网 百度文库 睡前故事 知乎 Wish
- python3
- requests
- scrapy
- spider
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BOSS Zhipin job scraper/crawler.
- boss-zhipin
- chrome-cdp
- spider
- boss-zhipin-scraper
Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.
- google-play
- google-play-api
- google-play-scraper
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
- playwright
- python
- scrapy
- scrapy-spider
Async Python 3.6+ web scraping micro-framework based on asyncio
- asyncio
- aiohttp
- asyncio-spider
- crawler
Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.