Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
132,241 visible repositories in the index
1–24 of 94 · Most stars
Scrapy, a fast high-level web crawling & scraping framework for Python.
🕵️♂️ Collect a dossier on a person by username from 6K websites
Proxy server to bypass Cloudflare protection
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Translated🤖 wukong-robot is a simple, flexible, and elegant Chinese voice dialogue robot / smart speaker project. It supports ChatGPT multi-turn conversation, and may also be the first open-source smart speaker project to support brain-computer interaction.
The Unofficial TikTok API Wrapper In Python
Open-Source Unified Vulnerability Management, DevSecOps & ASPM
Dark Web OSINT Tool
CISO Assistant is a one-stop-shop GRC platform for Risk Management, AppSec, Compliance & Audit, TPRM, BIA, Privacy, and Reporting. It supports 200+ global frameworks with automatic control mapping, including ISO 27001, NIST CSF, SOC 2, CIS, PCI DSS, NIS2, DORA, GDPR, HIPAA, CMMC, and more.
TranslatedAI Vtuber is a virtual streamer [Live2D/UE/xuniren] powered by [ChatterBot/ChatGPT/claude/langchain/chatglm/text-gen-webui/闻达 (Wenda)/千问 (Qianwen)/kimi/ollama]. It can interact with viewers in real time in live streams on [Bilibili/抖音 (Douyin)/快手 (Kuaishou)/微信视频号 (WeChat Channels)/拼多多 (Pinduoduo)/斗鱼 (Douyu)/YouTube/twitch/TikTok], or chat directly on the local machine. It uses TTS technology [edge-tts/VITS/elevenlabs/bark/bert-vits2/睿声 (Ruisheng)] to generate replies and can optionally change the voice with [so-vits-svc/DDSP-SVC]; commands work together with SD to draw images.
Scrape Facebook public pages without an API key
Errbot is a chatbot, a daemon that connects to your favorite chat service and bring your tools and some fun into the conversation.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
:musical_note: The original MusicBot for Discord (formerly SexualRhinoceros/MusicBot)
Advanced Multi-Featured Telegram UserBot, Built in Python Using Telethon lib.
Pycord is a modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python
An extensible message tunneling chat bot framework. Delivers messages to and from multiple platforms and remotely control your accounts.
splinter - python test framework for web applications
Telegram Bot to store Posts and Documents and it can Access by Special Links.
⛏⚽ Scrape soccer data from Club Elo, ESPN, FBref, Football-Data.co.uk, Sofascore, SoFIFA, Understat and WhoScored.
Financial Data Extraction from Investing.com with Python
A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!
Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess
The best open source bomber