TranslatedA new-generation web crawler platform that defines crawler workflows graphically, so you can build a crawler without writing code.
Repository Index
72,862 visible repositories in the index
1–3 of 3 · Most stars
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
- java
- webcrawling
- warc
- heritrix
WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.