Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
- crawling
- csharp
- elixir
- ffi
Repository Index
72,362 visible repositories in the index
121–133 of 133 · Most stars
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Price tracker monitors of products and alerts you when prices drop. Supported tiki.vn, shopee, lotte.vn, ... Built with firebase https://pricetrack.web.app
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
Pre-rendering for JavaScript websites that delivers SSR-level SEO, enhanced link previews, and performance via effortless middleware integration — ideal for PWAs, SPAs, and modern JS-driven apps, websites, and webpages
- meteor
- middleware
- crawler
- seo
Take HTML Snapshots for your Angular, Ember, Durandal or any JavaScript applications
Practical pinned gist which show your latest medium status 📌
- awesome-pinned-gists
- medium-stats
- github-gists
- github-action
Confluence Cloud to Markdown crawler with incremental updates and local link rewriting
SpiderMan,a base-on Broadway fast high-level web crawling & scraping framework for Elixir.
:arrow_forward: A .NET core library to handle the login to Backpack.tf. Backpack.tf is a trading site for Team Fortress 2, Counter-Strike: Global Offensive, and Dota 2. Community item pricing, item trading and stats, and much more.
- teamfortress2
- steam
- steambot
- steamweb
A package to searching courses - Super Mario Maker
- json
- mario-maker
- nodejs
- javascript
:repeat: An API wrapper for the TF2 Outpost platform. A platform to find great deals for your Team Fortress 2, Counter-Strike: Global Offensive and Dota 2 items with zero hassle.
- teamfortress2
- steam
- steambot
- steam-api
A package to fetching the maker profile - Super Mario Maker
- json
- mario-maker
- nodejs
- javascript