Crawly, a high-level web crawling & scraping framework for Elixir.
Repository Index
57,637 visible repositories in the index
25–40 of 40 · Most stars
A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user
- automation
- ecommerce
- filamentphp
- laravel
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Translated📚 WeChat Reading (微信读书) lossless full-book export tool — one-click export of dual-font, finely typeset PDF / EPUB / plain Markdown notes (compatible with Obsidian & Notion), solving the problem of missing characters when turning pages on the web version.
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Scrape any website, article or RSS/Atom Feed with ease!
- scrape
- elixir
- readability
- rss
Mac app for website auditing and crawling
- seo
- website-auditing
- swift
- mac
Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
- crawling
- csharp
- elixir
- ffi
MindAct: Community-Driven AI Agent for Claude, Obsidian, and RAG Knowledge Graphs
- activator
- analytics
- artificial-inteligence
- bigquery
An app to search startup jobs scraped from websites written in Elixir, Phoenix, React and styled-components.
Uses parselets and rwget to generate csv files from websites
Pre-rendering for JavaScript websites that delivers SSR-level SEO, enhanced link previews, and performance via effortless middleware integration — ideal for PWAs, SPAs, and modern JS-driven apps, websites, and webpages
- meteor
- middleware
- crawler
- seo
Website to Scrapping all writeup from http://ctftime.org/ and you can organize which to read first!
BNETDocs, the documentation and discussion website for Blizzard protocols
- battle-net
- blizzard
- bot
- diablo
Go package for scraping website recipes
- go
- golang
- recipe
- recipes
A PHP project to check if websites are linking to your website
- php
- competitors
- seo
- seo-optimization