Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
- java
- webcrawling
- warc
- heritrix
107,098 visible repositories in the index
1–4 of 4 · Most stars
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.
Open-source AI agents and workflow automation. Self-host or embed in your SaaS. Apache 2.0 alternative to n8n and Zapier.
Modern & Powerful Telegram-bot-api framework/library with custom features!