47,133 repositories indexed
1–14 of 14 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Command-line program to download image galleries and collections from several image hosting sites
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
The RSS feed for websites missing it
A Python module to bypass Cloudflare's anti-bot page.
💪 User-generated content for Git-powered websites
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of User Interface or API, also known as Headless browser. It can be used to run automated tests / snapshots and manipulate websites using Javascript.
Crawly, a high-level web crawling & scraping framework for Elixir.
Extract structured data from web sites. Web sites scraping.
🩵 Self-hosted & Fully-automated ActivityPub Bridge for Static Sites.
An app to search startup jobs scraped from websites written in Elixir, Phoenix, React and styled-components.
Uses parselets and rwget to generate csv files from websites
Pre-rendering for JavaScript websites that delivers SSR-level SEO, enhanced link previews, and performance via effortless middleware integration — ideal for PWAs, SPAs, and modern JS-driven apps, websites, and webpages
A PHP project to check if websites are linking to your website