Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.
- metadata
- article
- cheerio
- extract
Repository Index
79,350 visible repositories in the index
169–192 of 332 · Most stars
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
Generate and download e-books from online sources.
- lightnovel
- termux
- web-scraper
- console-app
日本电影元数据刮削器，配合kodi,emby,plex等本地媒体管理工具使用。可批量抓取，也可单个抓取。可抓取子目录下视频，多集视频（-cd1/-cd2）,带字幕作品（-c., -C.）。批量添加emby演员头像。
- jav
- jav-scraper
- emby
- kodi
:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
REST implementation of Django authentication system.
OSINT Tools for the Dark Web
- darkweb
- osint-tools
- search-engines
- scraping-tools
Advanced Privacy Browser Core with Unified Fingerprint Defense: Cloudflare, Akamai, Kasada, Shape, DataDome, PerimeterX, hCaptcha, FunCaptcha, Imperva, reCAPTCHA, ThreatMetrix, Adscore
- akamai
- cloudflare
- incapsula
- perimeterx
Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify.
- fingerprinting
- playwright
- puppeteer
- scraping
🔥 Visual workflow builder for AI agents powered by Firecrawl - drag-and-drop web scraping pipelines with real-time execution
A cloudflare verification bypass script for webscraping
- cloudflare-bypass
- bypass
- cloudflare
- cloudflare-bypass-script
Downloads and archives content from reddit
- imgur
- gfycat
- scraper
A CLI utility for taking screenshots of websites, recording video demos and scraping sites using JavaScript
- playwright
- playwright-python
- scraping
- screenshot-utility
A Python program to scrape secrets from GitHub through usage of a large repository of dorks.
A curated list of awesome puppeteer resources.
- puppeteer
- headless-chrome
- awesome
- awesome-list
invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
- datadome
- perimeterx
- akamai
- antidetect-browser
Learn Python for the next 30 (or so) Days.
This repositories contains a list of python scripts projects from beginner level advancing slowly. More code snippets to be added soon. feel free to clone this repo
- python-script
- webscrapping
- ascii-art
- beginner-project
Cr*ap Fixer
- crap
- fixer
- belim
Fully functional Pokerbot that works on PartyPoker, PokerStars and GGPoker, scraping tables with Open-CV (adaptable via gui) or neural network and making decisions based on a genetic algorithm and montecarlo simulation for poker equity calculation. Binaries can be downloaded with this link:
- poker
- pokerstars
- artificial-intelligence
Web Scraping Framework
- web-scraping
- http-client
- framework
- python
Scrape Twitter for Tweets
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Meridian cuts through news noise by scraping hundreds of sources, analyzing stories with AI, and delivering concise, personalized daily briefs.