Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Repository Index
47,133 repositories indexed
1–7 of 7 · Most stars
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
Turn any website into a structured API. Extract, automate, search and monitor the web.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
A scalable web crawler framework for Java.
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.
Practical pinned gist which show your latest medium status 📌