Ranked by Attention Score: stars gained over the last 7 complete days
10 repositories by Attention Score · Oct 27, 2026 – Nov 2, 2026 (UTC)
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Scrapy, a fast high-level web crawling & scraping framework for Python.
Turn any website into a structured API. Extract, automate, search and monitor the web.
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.
A Chrome DevTools Protocol driver for web automation and scraping.
Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Elegant Scraper and Crawler Framework for Golang
Crawly, a high-level web crawling & scraping framework for Elixir.