新浪微博爬虫(Scrapy、Redis)
Repository Index
107,098 visible repositories in the index
385–408 of 1,659 · Most stars
File Attachment toolkit for Ruby applications
- file-upload
- ruby
- rack
- storage
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
- amazon
- amazon-scraper
- python
- web-scraper
Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
Download the entire Wayback Machine archive for a given URL.
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
A serial device I/O tool
Scrapy+Splash for JavaScript integration
- scrapy
- headless-browsers
Translated[Captcha Recognition - Training] This project is based on CNN/ResNet/DenseNet+GRU/LSTM+CTC/CrossEntropy to realize verification code identification. This project is only for training the model.
- captcha-recognition
- tensorflow-tutorials
- ocr
- tensorflow
UI automation library for .Net
牛人快跑 - GeekGeekRun,求职找工作自动化工具,基于 Puppeteer、Electron。界面友好,简单易用!开源,免费!快速海投、自动开聊、已读不回提醒、职位信息抓取、不合适职位标记、僵尸职位清理;让你即刻变身成为一台冇得感情的求职机器🤖,找工作不内耗。宝宝不用动,妈妈全自动~祝你收获好Offer!
- automatic-control
- bosszhipin
- findjob
- jobsearch
Article publishing platform that automatically distributes your articles to various media channels
🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos, contact information, followers, following and much more.
- facebook-page-insights
- instagram-followers-scraper
- facebook-ads-metrics
- facebook-contact-info
A tiny, fast Rust CLI that drives a real browser over the Chrome DevTools Protocol — built for coding agents.
- rust
- agent
- browser-automation
- llm
最全最新中国省,市,地区json及sql数据
- china-regions
- china-location
- china
- china-city
Watch (parts of) webpages and get notified when something changes via e-mail, on your phone or via other means. Highly configurable.
- python
- webpage
- monitor
- automation
Twitter Text Libraries. This code is used at Twitter to tokenize and parse text to meet the expectations for what can be used on the platform.
- twitter-text
- tweet
- unicode
安卓平台自动化工具,无需 root。
- android
- automation
- hamibot
- autojs
Free Open Source Enterprise Grade RPA
- rpa
- workflow
- automation
- multiple-robots
Open Source AI trading agent that operates autonomously across 1000+ markets - Polymarket, Kalshi, Binance, Hyperliquid, Solana DEXs, 5 EVM chains. Scans for edge, executes instantly, manages risk while you sleep. Agent commerce protocol for machine-to-machine payments. Self-hosted. Built on Claude.
A service daemon to run Scrapy spiders
AI-powered Smart Robotic Process Automation 🤖
🍥 Bilibili 用户爬虫
- bilibili
- python3
- mysql
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.