Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
Repository Index
47,133 repositories indexed
1–7 of 7 · Most stars
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
:scissors: High performance, multi-threaded image scraper
A Python Package for the Google Chrome Dev Protocol [threading base]
- chrome-debugging-protocol
- python
- chrome
- headless
A Python feed reader library.
- rss
- rss-reader
- rss-aggregator
- rss-feed
Python toolkit for discovering, monitoring, and controlling Tasmota devices via MQTT with CLI utilities and ready-to-use multi-arch Docker images
- automation
- cli
- control
- device-management