Repository Index

The fast, flexible, and elegant library for parsing and manipulating HTML and XML.

• cheerio
• jquery
• htmlparser2
• dom

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

• web-scraping
• web-crawling
• npm
• headless-chrome

Crawl a site to generate knowledge files to create your own custom GPT from a URL

• ai

WebDriver Classic proxy for automating Firefox through Marionette

• webdriver
• geckodriver
• firefox
• rust

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

• crawler
• javascript
• spider
• extract-data

An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.

• telegram
• archive
• wayback-machine
• save-the-internet

querySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer

• webcomponents
• shadow-dom
• javascript
• puppeteer

Telegram Bot library written in V.

• v
• vlang
• telegram
• bot

Python type wrappers for Chrome DevTools Protocol (CDP)

Trio driver for Chrome DevTools Protocol (CDP)

A hilarious Sinatra-based Markov bot for Slack.

• markov-bot
• sinatra
• heroku
• ruby

atwSCADA — Free SCADA Software in a Single HTML File | IEC 61850, OPC UA, Modbus TCP | 53 Function Blocks, 65 Graphic Elements | No Installation | Download Free Demo

• digital-substation
• hmi
• iec61850
• industrial-automation

An headline and link puller for Reddit and its various subreddits

A package to searching courses - Super Mario Maker

• json
• mario-maker
• nodejs
• javascript

A package to fetching the maker profile - Super Mario Maker

• json
• mario-maker
• nodejs
• javascript

Sitecore Content Hub Asset Importer. Allows importing image assets from various data sources into Sitecore Content Hub.