新闻网页正文通用抽取器 Beta 版.
- python3
- webcrawler
- webspider
110,088 visible repositories in the index
25–48 of 88 · Most stars
新闻网页正文通用抽取器 Beta 版.
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.
Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.
Python package for scraping recipes data
Automatically extract body content (and other cool stuff) from an html document
Browser and desktop automation with an MCP server for AI assistants. Record macros or write JavaScript to automate websites, extract data, and control desktop apps with OCR and image recognition.
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
NewPipe's core library for extracting data from streaming sites
To extract article from given URL
Financial Data Extraction from Investing.com with Python
A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.
LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping
This package is designed to bypass puppeteer's bot-detecting captchas such as Cloudflare. It acts like a real browser and can be managed with puppeteer.