Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
Repository Index
58,632 visible repositories in the index
1–24 of 59 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Your lightweight open-source personal agent.
🕵️♂️ Collect a dossier on a person by username from 6K websites
- osint
- social-network
- identification
- socmint
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring— MCP included! all for free or enjoy our SaaS plan!
- website-monitor
- website-monitoring
- change-detection
- monitoring
Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
We have made you a wrapper you can't refuse
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
An API wrapper for Discord written in Python.
- discord
- discord-api
- python
- python-3
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
Download pictures (or videos) along with their captions and other metadata from Instagram.
- instagram-client
- instagram-feed
- downloader
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
PersonaPlex code.
Translated:orange_book: Chinese Xinhua Dictionary database. Includes xiehouyu (two-part allegorical sayings), idioms, words, and Chinese characters.
- data
- scraper
- chinese-traditional
- python3
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
- apify
- automation
- beautifulsoup
- crawler
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
- python
- python-script
- image-download
- google-images
Android Uiautomator2 Python Wrapper
- python
- uiautomator
- test
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
TranslatedGenerate all kinds of short videos in batches with one click using AI, automatically remix short videos in batches, and automatically publish videos to Douyin, Kuaishou, Xiaohongshu, and Video Accounts (WeChat Channels). Making money has never been this easy! Supports local voice models chatTTS, fasterwhisper, GPTSoVITS, and cloud voice services: Azure, Alibaba Cloud, Tencent Cloud. Supports Stable diffusion and comfyUI for direct AI image generation. Generate short videos with one click using AI LLM,print money together! support:chatTTS,faster-whisper,GPTSoVITS,Azure,tencent Cloud,Ali Cloud.
🔥 The fastest and powerful Python library for Instagram Private API 2026 with HikerAPI SaaS
- instagram-api
- instagram-client
- instagram-downloader
MaiSaka, an LLM-based intelligent agent, is a digital lifeform devoted to understanding you and interacting in the style of a real human. She does not pursue perfection, nor does she seek efficiency; instead, she values warmth, authenticity, and genuine connection.
- chat
- llm
- python
- qq-bot
A Python module to bypass Cloudflare's anti-bot page.
- cloudflare
- cloudflare-bypass
- cloudflare-scrape
- anti-bot-page