Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Repository Index
55,640 visible repositories in the index
1–4 of 4 · Most stars
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants
Appium Native Crawler CLI - Features include: Screenshots, Performance, Accessibility Detection, Google Translate, Applitools, Monkey Tester