Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
Repository Index
47,133 repositories indexed
1–8 of 8 · Most stars
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Python ProxyPool for web spider
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Incredibly fast crawler designed for OSINT.
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
Credential Phish Analysis and Automation
SimpleCRON is a time-based task scheduling program inspired by the well-known CRON program for Unix systems.