Sign in

Repository Index

132,241 visible repositories in the index

Tags: Web scraper
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–16 of 16 · Most stars

⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file

  • save-the-internet
  • procrastination
  • its-mine
  • no-more-404

AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

  • aitools
  • crawl
  • scrape
  • serp

Real time google image scraper. The process is automated by sending HTTP requests to retrieve image data which is then parsed and saved.

  • google-image-scraper
  • google-images-crawler
  • image-scrape
  • image-scraper

A system of bots that collects clips automatically via custom made filters, lets you easily browse these clips, and puts them together into a compilation video ready to be uploaded straight to any social media platform. Full VPS support is provided, along with an accounts system so multiple users can use the bot at once. This bot is split up into three separate programs. The server. The client. The video generator. These programs perform different functions that when combined creates a very powerful system for auto generating compilation videos.

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

  • xquery
  • xml
  • html
  • json