TranslatedA visual no-code/code-free web crawler/spider. 易采集 (EasySpider, "easy collection"): a visual browser automation testing/data collection/web crawler software that lets you design and execute crawler tasks graphically without code. Alias: ServiceWrapper, an intelligent service wrapping system for Web applications.
Repository Index
47,133 repositories indexed
1–24 of 25 · Most stars
Elegant Scraper and Crawler Framework for Golang
Python ProxyPool for web spider
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
Incredibly fast crawler designed for OSINT.
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
TranslatedA new-generation web crawler platform that defines crawler workflows graphically, so you can build a crawler without writing code.
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Dark Web OSINT Tool
- tor
- tor-network
- deepweb
- dark-web
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Async Python 3.6+ web scraping micro-framework based on asyncio
- asyncio
- aiohttp
- asyncio-spider
- crawler
Anemone web-spider framework
A configurable and extensible PHP web spider
Crawly, a high-level web crawling & scraping framework for Elixir.
A high performance web crawler / scraper in Elixir.
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
A AI Browser run any site JavaScript from Python with the real Firefox engine. Headless SpiderMonkey + Necko for automation and scraping.
- anti-bot
- automation
- firefox
- headless-browser
The unix-way web crawler
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Web spider as a service, spider on serverless
- web-spider
- serverless
- javascript
- api
Pre-rendering for JavaScript websites that delivers SSR-level SEO, enhanced link previews, and performance via effortless middleware integration — ideal for PWAs, SPAs, and modern JS-driven apps, websites, and webpages
- meteor
- middleware
- crawler
- seo
SpiderMan,a base-on Broadway fast high-level web crawling & scraping framework for Elixir.