基于 scrapy-redis 的通用分布式爬虫框架
- spiderman
- scapy-redis
- kafka
- hive
142,273 visible repositories in the index
385–408 of 452 · Most stars
基于 scrapy-redis 的通用分布式爬虫框架
Cryptocurrency historical price data library in Python. Data from https://coinmarketcap.com.
LinkedIn Scraper (currently working 2020)
Plex metadata scraper for Audiobooks
Google Reviews API integration and scalable Google Reviews Scraper for extracting ratings, review text, and business insights from Google Maps using a structured Google Review API.
⭐️ A proxy scraper made using Protractor | Proxy list Updates every three hour 🔥
scrape proxies from more than 5 different sources and check which ones are still alive
n8n node for browser automation using Puppeteer
Fetch user's data across social media
A universal web-util for PHP.
Free Web Scraping Tool with Java
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.
The REST API for Jikan
Undetected web-scraping & seamless HTML parsing in Python!
Javascript scraping module based on puppeteer for many different search engines...
OpenClaw-inspired autonomous AI agent built entirely in n8n. Adaptive RAG-powered memory, Skills via MCP templates, Expert Agents with delegated sub-agents, proactive task management, media understanding - self-hosted with one setup script
Powerful and versatile game scraper written in c++
Scrapy Extension for monitoring spiders execution.
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
NBA Stats API via Basketball Reference
Simple yet powerful automation stuffs.
a reliable high-level web crawling & scraping framework for Node.js.
Jekyll-based static site for The Programming Historian
Web Scraping with GPT-4 Vision API and Puppeteer