Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
TranslatedXiaohongshu notes | comment crawler, Douyin videos | comment crawler, Kuaishou videos | comment crawler, Bilibili videos | comment crawler, Weibo posts | comment crawler, Baidu Tieba posts | Baidu Tieba comment and reply crawler | Zhihu Q&A and articles | comment crawler
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Python ProxyPool for web spider
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Incredibly fast crawler designed for OSINT.
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
TranslatedAn open-source AI agent for social media — discover trends, create content, publish everywhere, and learn what works across Xiaohongshu, Douyin, Zhihu, Bilibili, and more.🎨
- agent
- agent-skill
- content-automation
- content-creation
A repo-native coding-agent loop harness for Claude, Codex, Copilot, and other CLI agents. Agents can edit. Gates decide what lands. You set the plan in motion. The agents take a prompt, and each iteration must update specs and commit through guardrails.
A Python package for automating VideoIPath configuration workflows.
- videoipath
- media-over-ip
- python
- configuration-management