新浪微博爬虫（Scrapy、Redis）
Repository Index
71,235 visible repositories in the index
241–264 of 564 · Most stars
Chromium Binary for AWS Lambda and Google Cloud Functions
- aws-lambda
- puppeteer
- chromium
- serverless
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
- amazon
- amazon-scraper
- python
- web-scraper
Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
TrendPublish: 全自动 AI 内容生成与发布系统 | 微信公众号自动化 | 多源数据抓取 (Twitter/X、网站) | DeepseekAI、千问、讯飞模型 | 智能内容分析排序 | 定时发布 | 多模板支持 | Node.js | TypeScript | AI 技术趋势跟踪工具
- ai
- weixin
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
The open-source Manychat alternative
Scrapy+Splash for JavaScript integration
- scrapy
- headless-browsers
A serial device I/O tool
TranslatedMulti-platform content monitoring, collection, and reposting: one web panel to manage Douyin / Xiaohongshu / Kuaishou / WeChat Channels / TikTok
- automation
- content-monitoring
- crawler
- douyin
Article publishing platform that automatically distributes your articles to various media channels
🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos, contact information, followers, following and much more.
- facebook-page-insights
- instagram-followers-scraper
- facebook-ads-metrics
- facebook-contact-info
Turn any website into an API your agent can call. Learns from the site's own frontend, heals itself when the site changes.
最全最新中国省，市，地区json及sql数据
- china-regions
- china-location
- china
- china-city
Open Source AI trading agent that operates autonomously across 1000+ markets - Polymarket, Kalshi, Binance, Hyperliquid, Solana DEXs, 5 EVM chains. Scans for edge, executes instantly, manages risk while you sleep. Agent commerce protocol for machine-to-machine payments. Self-hosted. Built on Claude.
A service daemon to run Scrapy spiders
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
A browser testing and web crawling library for PHP and Symfony
Automatically crawl arXiv papers daily and summarize them using AI. Illustrating them using GitHub Pages.
Make your job hunt easy by automating your application process with this Auto Applier
- python
- selenium
- webscraping
- automation
Perplexity Scraper Track brand mentions, analyze rankings, and gain competitor intelligence from Perplexity. Get started in minutes.
- ai-scraper
- llm-api
- llm-scraper
- llm-scraping
📲 An agent for sourcing, curating, and scheduling social media posts with human-in-the-loop.
Node.js scraper to get data from Google Play
A scriptable browser like PhantomJS, based on Firefox
- slimerjs
- casperjs
- firefox
- javascript