:scissors: High performance, multi-threaded image scraper
Repository Index
47,133 repositories indexed
73–96 of 130 · Most stars
Extracts information about web pages, like youtube videos, twitter statuses or blog articles.
- php
- embed
- oembed
- opengraph
A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user
- automation
- ecommerce
- filamentphp
- laravel
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
A AI Browser run any site JavaScript from Python with the real Firefox engine. Headless SpiderMonkey + Necko for automation and scraping.
- anti-bot
- automation
- firefox
- headless-browser
bike sharing + python = pybikes
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
The unofficial HLTV Node.js API
Translated✈️ Self-hosted multi-channel airfare price monitoring: flight detail collection → DingTalk rich-media push → four-layer strong alerts when the target price is met. Domestic routes and international routes (destinations on five continents) share the same collect-judge-push pipeline; data stays on your own machine, all configuration is done on a web page, open source and free.
- crawler
- dingtalk
- flight
- playwright
Get video info from Dailymotion, Vimeo, Wistia, and YouTube URLs.
Functional HTML scraping and rewriting with CSS in OCaml
WarcDB: Web crawl data as SQLite databases.
TranslatedAn Agent Skill for scraping the complete comment section of a single Douyin video (top-level comments + nested replies): it copies the page's real requests inside a logged-in Chrome and leaves the signing to the page's SDK
- agent-skill
- claude-code
- comments-scraper
- douyin
Zotero extension that combats link rot by archiving webpages and journal articles.
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
The unix-way web crawler
Scrape any website, article or RSS/Atom Feed with ease!
- scrape
- elixir
- readability
- rss
Send Sir Perceval on a quest to retrieve and gather data from software repositories.
- grimoirelab
- perceval
- data-mining
- data-fetching
An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
DocSearch - Scraper
- algolia
- docsearch
- documentation
- scraper
Readability is Elixir library for extracting and curating articles.
- readability
- webpage
- parser
- elixir
One-link downloader for the Mac (videos, documents, images, articles, web pages), with optional automatic YouTube monitoring that translates and subtitles new videos.
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
- golang
- unmarshall
- goquery
- selectors
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping