Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
- xquery
- xml
- html
- json
Repository Index
47,133 repositories indexed
121–144 of 207 · Most stars
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Extracts information about web pages, like youtube videos, twitter statuses or blog articles.
- php
- embed
- oembed
- opengraph
A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user
- automation
- ecommerce
- filamentphp
- laravel
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Slack real time messaging and web API client in Elixir
A Discord API wrapper library made in C
Lightweight Node.js library for building Slack Block Kit UIs, with a declarative syntax inspired by SwiftUI.
Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.
- webkit
- headless-browsers
- erik
- swift
Slack Developer Kit (including Bolt for Java) for any JVM language
Tor Browser automation with Selenium.
- selenium
- tor-browser
- tor
- automation
Cloudlog is a web-based amateur radio logging application built with PHP and MySQL. It supports a wide range of station logging activities, from HF to microwave, and integrates with companion tools for CAT control and automation.
- cloudlog
- ham-radio
- amateur-radio
- php
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.
A port of puppeteer running on Deno
WarcDB: Web crawl data as SQLite databases.
Tinode web chat using React
- chat
- chat-application
- chatapp
- react
Zotero extension that combats link rot by archiving webpages and journal articles.
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
TranslatedDeploy and manage AstrBot and NapCat locally on Android phones, with a built-in PRoot Ubuntu runtime environment. Supports a native management interface, WebUI, logs, backup and restore, and independent updates, with no Root required.
The unix-way web crawler
Scrape any website, article or RSS/Atom Feed with ease!
- scrape
- elixir
- readability
- rss
A dead simple RSS Feed with Discord Webhooks
Readability is Elixir library for extracting and curating articles.
- readability
- webpage
- parser
- elixir