Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
Repository Index
88,073 visible repositories in the index
1–20 of 20 · Most stars
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit
Android Full-Stack Device Control Platform: WebRTC/H.264 remote desktop, UI/OCR/image-matching automation, one-click MITM, built-in Frida, proxy/VPN/frp/P2P networking, MCP/Agent, 160+ APIs, designed for multi-device clusters and engineered deployments.
- adb
- android
- frida
- uiautomation
Lighter web automation with Python
- selenium
- selenium-python
- python
- python3
The first AI agent that builds permissionless integrations through reverse engineering platforms' internal APIs.
A library that scrapes Linkedin for user data
- linkedin-scraper
- scraper
- profile
🐍Python 3 wrapper of Microsoft UIAutomation. Support UIAutomation for MFC, WindowsForm, WPF, Modern UI(Metro UI), Qt, IE, Firefox, Chrome ...
invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
- datadome
- perimeterx
- akamai
- antidetect-browser
💬 RasaGPT is the first headless LLM chatbot platform built on top of Rasa and Langchain. Built w/ Rasa, FastAPI, Langchain, LlamaIndex, SQLModel, pgvector, ngrok, telegram
Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.
For EHRs, payer portals, and any app your team clicks through. A person or an agent shows the task; OpenAdapt turns it into a local program that repeats the steps with no model calls, checks each save, and stops when something doesn't match.
- desktop-automation
- gui-automation
- local-first
- workflow-automation
the first third-party instant messaging client for Google Hangouts
- python
- asyncio
- python-3
- chat
A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!
- chrome
- chrome-devtools-protocol
- python
- webscraping
🎭 Playwright integration for Scrapy
- playwright
- playwright-python
- scrapy
- python
A web privacy measurement framework
ARES - authorized red-team engagement automation with dashboard, campaign scope, module orchestration, OPSEC controls, encrypted vault, and reporting.
- active-directory
- cloud-security
- defensive-validation
- fastapi
Z-Hound — a CSV-first collector that finds public threat reports and extracts structured IoCs without a database, LLM, or API key.
Self hosted, real-time digital human agent platform. Build voice-first AI agents with WebRTC, persona memory, tools, RAG, and optional digital-human video.
Website to Scrapping all writeup from http://ctftime.org/ and you can organize which to read first!