Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
Repository Index
71,372 visible repositories in the index
1–24 of 48 · Most stars
Your lightweight open-source personal agent.
TranslatedA complete and graceful API for Wechat. WeChat personal account API, WeChat robot, and command-line WeChat; you can customize a personal account robot in thirty lines.
Proxy server to bypass Cloudflare protection
- puppeteer
- cloudflare
- proxy
- chrome
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Generic automation framework for acceptance testing and RPA
- robotframework
- testing
- automation
- testautomation
Android Full-Stack Device Control Platform: WebRTC/H.264 remote desktop, UI/OCR/image-matching automation, one-click MITM, built-in Frida, proxy/VPN/frp/P2P networking, MCP/Agent, 160+ APIs, designed for multi-device clusters and engineered deployments.
- adb
- android
- frida
- uiautomation
TranslatedA self-media customer acquisition agent tailor-made for OPC/small, medium and micro enterprises
- mas
- multi-agent
- multi-agent-systems
- makemoney
TranslatedAsynchronous multi-platform chatbot framework written in Python
- nonebot
- cqhttp
- bot
Translated🤖 wukong-robot is a simple, flexible, and elegant Chinese voice dialogue robot / smart speaker project. It supports ChatGPT multi-turn conversation, and may also be the first open-source smart speaker project to support brain-computer interaction.
With EasyFNBot you can easily create you own Fortnite Lobby Bot in less than 5 minutes which will be online forever!
- fortnitepy
- fortnite
- python
- easy-to-use
The Unofficial TikTok API Wrapper In Python
- tiktok-api
- python
- api
- trending
TranslatedHands-on 🐍 web crawlers for many kinds of websites and e-commerce data 🕷. Includes 🕸: Taobao products, WeChat official accounts, Dianping, Qichacha, job recruitment sites, Xianyu, Alibaba tasks, Cnblogs, Weibo, Baidu Tieba, Douban movies, Baotu (ibaotu.com), Quanjing (quanjing.com), Douban music, a provincial drug administration, Sohu News, machine learning text collection, FOFA asset collection, Autohome, National Bureau of Statistics, Baidu keyword indexed-page counts, spider pool directories, Toutiao (Today's Headlines), Douban movie reviews, Ctrip, Xiaomi App Store, Anjuke, Tujia homestays ❤️❤️❤️. WeChat crawler showcase project:
- python3
- crawler
- baidu-tieba
- taobao-spider
Open-Source Unified Vulnerability Management, DevSecOps & ASPM
Dark Web OSINT Tool
- tor
- tor-network
- deepweb
- dark-web
The first AI agent that builds permissionless integrations through reverse engineering platforms' internal APIs.
Jobs scraper library for LinkedIn, Indeed, Glassdoor, ZipRecruiter, Bayt, Naukri, BDJobs & HelloWork
- indeed
- ziprecruiter
- jobs-scraper
TranslatedWeRoBot is a WeChat official account development framework
- python
- weixin
- wechat-framework
CISO Assistant is a one-stop-shop GRC platform for Risk Management, AppSec, Compliance & Audit, TPRM, BIA, Privacy, and Reporting. It supports 200+ global frameworks with automatic control mapping, including ISO 27001, NIST CSF, SOC 2, CIS, PCI DSS, NIS2, DORA, GDPR, HIPAA, CMMC, and more.
- compliance
- grc
- soc2
- gdpr
A list of AI agents and robots to block.
Open-source MCP server for LinkedIn. Give Claude and any MCP-compatible AI agent access to profiles, companies, jobs, and messages.
- linkedin-scraper
- mcp
- mcp-server
Hook and simulate global keyboard events on Windows and Linux.
- python
- keyboard-events
- keyboard-hooks
- keyboard-state
Use Google News Scraper API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.
- google-news-scraper
- google-scraper
- google-scraper-api
- news-scraper
Scrape Facebook public pages without an API key
- scraping
- facebook-scraping
- facebook-scraper