Repository Index

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

• agent-infrastructure
• ai-agent
• ai-search
• automation

Python scraper based on AI

• scraping
• scraping-python
• llm
• web-crawler

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

• pocket
• wget
• browser-bookmarks
• pinboard

TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.

• tiktok
• douyin
• scraper
• crawler

日常交给三月七，开拓留给你。《崩坏：星穹铁道》开源自动化助手。

• honkai-star-rail
• star-rail
• march7thassistant

WarcDB: Web crawl data as SQLite databases.

• crawling
• sqlite
• warc
• cli

DocSearch - Scraper

• algolia
• docsearch
• documentation
• scraper

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

• gpt
• openai
• context
• embedding

micrOS - mini asynchronous automation OS for DIY projects.

• micropython
• iot-platform
• interrupts
• socket

MaiBot 百度搜索插件，支持搜索命令与大模型 Tool，接入百度千帆官方 API。

Intelligent web page comparison tool with Wayback Machine support and visual regression testing

• visual-comparison
• wayback-machine
• web-scraping
• diff

Python toolkit for discovering, monitoring, and controlling Tasmota devices via MQTT with CLI utilities and ready-to-use multi-arch Docker images

• automation
• cli
• control
• device-management