Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
Repository Index
76,855 visible repositories in the index
1–24 of 248 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
Free, open source crypto trading bot
🕵️♂️ Collect a dossier on a person by username from 6K websites
- osint
- social-network
- identification
- socmint
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!
- website-monitor
- website-monitoring
- change-detection
- monitoring
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
- anti-detect
- bot-detection
- browser-automation
- chromium
Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
Automate the process of making money online.
- automation
- cli
- json
- money
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
An Open Phone Agent Model & Framework. Unlocking the AI Phone for Everyone
- agent
- phone-use-agent
The premier source of truth powering network automation. Open source under Apache 2. Try NetBox Cloud free: https://netboxlabs.com/products/free-netbox-cloud/
- dcim
- ipam
- netbox
- network-automation
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
- nlp
- machine-learning
- machine-learning-library
- bot
Command-line program to download image galleries and collections from several image hosting sites
- gallery
- downloader
- pixiv
- danbooru
Translated🤖 DIY-able multimodal AI chatbot | 🚀 Quickly connects to chat platforms such as WeChat, QQ, Telegram, and more | 🦈 Supports DeepSeek, Grok, Claude, Ollama, Gemini, OpenAI | Workflow system, web search, AI image generation, persona tuning, virtual maid, voice conversation |
TranslatedProduction-grade platform for building agentic IM bots - Agent, knowledge base orchestration, plugin system / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信 (Qiye Weixin, "Enterprise WeChat"), 企微智能机器人 (Qiwei Zhineng Jiqiren, "Enterprise WeChat smart bot"), 公众号 (Gongzhonghao, "official account")) / 飞书 (Feishu, "flying book") / 钉钉 (Dingding, "nail nail") / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow
TranslatedAutomatically upload videos to social media: Douyin, Xiaohongshu, Shipinhao (WeChat Channels), TikTok, YouTube, Bilibili
- bilibili
- douyin
- tiktok
- xiaohongshu
🚀 PR Agent: The Original Open-Source PR Reviewer. This project is not the Qodo free tier.
- openai
- pull-request
- code-review
- pull-requests
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit