1.firecrawl / firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
+38,224 stars
from 159,452 · score 10.55
Trending repositories
Ranked by Attention Score: stars gained over the last 90 complete days
24 repositories by Attention Score · Aug 5, 2026 – Nov 2, 2026 (UTC)
2.D4Vinci / Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
+19,806 stars
from 71,938 · score 9.89
3.jo-inc / camofox-browser
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
+3,886 stars
from 8,216 · score 8.27
4.soxoj / maigret
🕵️♂️ Collect a dossier on a person by username from 6K websites
+3,741 stars
from 35,865 · score 8.23
5.ScrapeGraphAI / Scrapegraph-ai
Python scraper based on AI
+3,556 stars
from 28,866 · score 8.18
6.scrapy / scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
+1,744 stars
from 63,418 · score 7.46
7.apify / crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
+1,228 stars
from 25,083 · score 7.11
8.feder-cr / invisible_playwright
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
+739 stars
from 1,809 · score 6.61
9.adbar / trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
+665 stars
from 6,385 · score 6.50
10.gocolly / colly
Elegant Scraper and Crawler Framework for Golang
+225 stars
from 25,346 · score 5.42
11.ultrafunkamsterdam / undetected-chromedriver
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
+182 stars
from 12,713 · score 5.21
12.neon-mmd / websurfx
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine
+65 stars
from 1,158 · score 4.19
13.playwright-php / playwright
Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!
+52 stars
from 203 · score 3.97
14.transitive-bullshit / awesome-puppeteer
A curated list of awesome puppeteer resources.
+24 stars
from 2,558 · score 3.22
15.code4craft / webmagic
A scalable web crawler framework for Java.
+20 stars
from 11,650 · score 3.04
16.online-judge-tools / oj
Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.
+15 stars
from 1,161 · score 2.77
17.elixir-crawly / crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
+15 stars
from 1,112 · score 2.77
18.psf / requests-html
Pythonic HTML Parsing for Humans™
+14 stars
from 13,786 · score 2.71
19.symfony / panther
A browser testing and web crawling library for PHP and Symfony
+13 stars
from 3,057 · score 2.64
20.sparklemotion / mechanize
Mechanize is a ruby library that makes automated web interaction easy.
+5 stars
from 4,440 · score 1.79
21.php-embed / Embed
Get info from any web service or page
+4 stars
from 2,135 · score 1.61
22.sananth12 / ImageScraper
:scissors: High performance, multi-threaded image scraper
+3 stars
from 771 · score 1.39
23.slotix / dataflowkit
Extract structured data from web sites. Web sites scraping.
+3 stars
from 711 · score 1.39
24.aantron / lambdasoup
Functional HTML scraping and rewriting with CSS in OCaml
+1 stars
from 408 · score 0.69