TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Repository Index
47,133 repositories indexed
1–5 of 5 · Most stars
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
A scalable, mature and versatile web crawler based on Apache Storm
- web-crawler
- apache-storm
- distributed
- java
Distributed Automation for XNAT
- xnat
- python
- dax