1.firecrawl / firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
+83,933 stars
from 113,743 · score 11.34
Trending repositories
Ranked by Attention Score: stars gained over the last 180 complete days
23 repositories by Attention Score · May 7, 2026 – Nov 2, 2026 (UTC)
2.D4Vinci / Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
+46,038 stars
from 45,706 · score 10.74
3.soxoj / maigret
🕵️♂️ Collect a dossier on a person by username from 6K websites
+13,971 stars
from 25,635 · score 9.54
4.ScrapeGraphAI / Scrapegraph-ai
Python scraper based on AI
+8,199 stars
from 24,223 · score 9.01
5.jo-inc / camofox-browser
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
+8,038 stars
from 4,064 · score 8.99
6.scrapy / scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
+4,050 stars
from 61,112 · score 8.31
7.apify / crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
+3,411 stars
from 22,900 · score 8.14
8.adbar / trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
+1,227 stars
from 5,823 · score 7.11
9.ultrafunkamsterdam / undetected-chromedriver
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
+441 stars
from 12,454 · score 6.09
10.gocolly / colly
Elegant Scraper and Crawler Framework for Golang
+435 stars
from 25,136 · score 6.08
11.neon-mmd / websurfx
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine
+140 stars
from 1,083 · score 4.95
12.playwright-php / playwright
Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!
+111 stars
from 144 · score 4.72
13.code4craft / webmagic
A scalable web crawler framework for Java.
+58 stars
from 11,612 · score 4.08
14.elixir-crawly / crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
+49 stars
from 1,078 · score 3.91
15.transitive-bullshit / awesome-puppeteer
A curated list of awesome puppeteer resources.
+42 stars
from 2,540 · score 3.76
16.psf / requests-html
Pythonic HTML Parsing for Humans™
+36 stars
from 13,764 · score 3.61
17.online-judge-tools / oj
Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.
+33 stars
from 1,143 · score 3.53
18.symfony / panther
A browser testing and web crawling library for PHP and Symfony
+26 stars
from 3,044 · score 3.30
19.sparklemotion / mechanize
Mechanize is a ruby library that makes automated web interaction easy.
+10 stars
from 4,435 · score 2.40
20.slotix / dataflowkit
Extract structured data from web sites. Web sites scraping.
+8 stars
from 706 · score 2.20
21.php-embed / Embed
Get info from any web service or page
+7 stars
from 2,132 · score 2.08
22.sananth12 / ImageScraper
:scissors: High performance, multi-threaded image scraper
+4 stars
from 770 · score 1.61
23.Anonyfox / elixir-scrape
Scrape any website, article or RSS/Atom Feed with ease!
+1 stars
from 335 · score 0.69