A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Repository Index
58,136 visible repositories in the index
193–216 of 1,009 · Most stars
TranslatedPython API for JMComic | Provides a Python API for accessing 禁漫天堂 (JMComic), supporting both the web and mobile clients | 禁漫天堂 (JMComic) GitHub Actions downloader🚀
- 18comic
- crawler
- github-actions
- pypi
🔥 Open Source Browser API for AI Agents & Apps. Steel Browser is a batteries-included browser sandbox that lets you automate the web without worrying about infrastructure.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
- golang
- google-maps-scraping
- web-scraper
- web-scraping
TranslatedGet the real streaming URLs (live stream sources) and danmaku (live comments) of 58 live streaming platforms, including Douyu, Huya, Bilibili, Douyin, and Kuaishou. The live stream sources can be played in players such as PotPlayer and flv.js.
- douyu
- huya
- bilibili
- python
Go library for the WhatsApp web multidevice API
- go
- golang
- whatsapp-web
TranslatedAsynchronous multi-platform chatbot framework written in Python
- nonebot
- cqhttp
- bot
generate release PRs based on the conventionalcommits.org spec
OpenAI + LINE + Vercel = GPT AI Assistant
bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
- mattermost
- xmpp
- telegram
- gitter
Pholcus is a distributed high-concurrency crawler software written in pure golang
- spider
- crowler
The first open-source agent skills builder. Define skills by vibe workflow, run on Claude Code, Cursor, Codex & more. Build Clawdbot 🦞· APIs for Lovable · Bots for Slack & Lark/Feishu · Skills are infrastructure, not prompts.
TranslatedDamai automatic ticket grabbing, supports selection of attendees, cities, date sessions, and prices
WebDriver Classic proxy for automating Firefox through Marionette
- webdriver
- geckodriver
- firefox
- rust
TranslatedAutomation for the Windows version of the WeChat client (not the web version), which can implement simple sending and receiving of WeChat messages, a simple WeChat bot
The AI local media metadata crawler
- emby
- kodi
- plex
- python
💯 Teach puppeteer new tricks through plugins.
- puppeteer
- chrome
- automation
- headless-chrome
TranslatedWeb WeChat API, including a terminal version of WeChat and a WeChat bot
One-stop Proxies Crawling and Aggregation Platform
- proxypool
Desktop Telegram client with good customization and Ghost mode.
- anti-recall
- anti-revoke
- ghost-mode
- telegram
TranslatedGenerate all kinds of short videos in batches with one click using AI, automatically remix short videos in batches, and automatically publish videos to Douyin, Kuaishou, Xiaohongshu, and Video Accounts (WeChat Channels). Making money has never been this easy! Supports local voice models chatTTS, fasterwhisper, GPTSoVITS, and cloud voice services: Azure, Alibaba Cloud, Tencent Cloud. Supports Stable diffusion and comfyUI for direct AI image generation. Generate short videos with one click using AI LLM,print money together! support:chatTTS,faster-whisper,GPTSoVITS,Azure,tencent Cloud,Ali Cloud.
TranslatedMore and more websites have anti-crawler features: some hide key data in images, and some use inhuman CAPTCHAs. This is a repository of anti-anti-crawler code, built to improve skills by contending with websites of different characteristics (with no malicious intent). (Submissions of websites that are hard to scrape are welcome.) (The project is paused due to work reasons.)
- python
- spider
- geek
TranslatedWeChat HOOK, WeChat robot, wxhook, database decryption, WeChat official account collection, WeChat official account crawler, WeCom (Enterprise WeChat) HOOK
- wechatrobot
- wechatapi
- wxhook
🤖 Keep your forks up-to-date via automated PRs
- probot
- sync
- fork
- upstream