57,138 visible repositories in the index
1–24 of 110 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Scrapy, a fast high-level web crawling & scraping framework for Python.
Translated:traffic_light:Mini Program Radar: an AI-driven tool for mini program technology selection, trend tracking, and migration diagnosis
Your lightweight open-source personal agent.
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
Make Any Website into CLI & Use your logged-in browser by AI agent.
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
TranslatedA one-click tool for the daily tasks of Arknights, supporting all clients. (Arknights assistant: one click to finish all your daily tasks!)
Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app
TranslatedA multi-task real-time/scheduled monitoring and intelligent analysis system for Xianyu (闲鱼), built with Playwright and AI, and equipped with a full-featured back-end management UI. It helps users find the products they like among the vast number of items on Xianyu.
Translated🤖 DIY-able multimodal AI chatbot | 🚀 Quickly connects to chat platforms such as WeChat, QQ, Telegram, and more | 🦈 Supports DeepSeek, Grok, Claude, Ollama, Gemini, OpenAI | Workflow system, web search, AI image generation, persona tuning, virtual maid, voice conversation |
Turn any website into a structured API. Extract, automate, search and monitor the web.
A lightweight, extendable front-end developer tool for mobile web page.
CLI for common Playwright actions. Record and generate Playwright code, inspect selectors and take screenshots.
Python version of the Playwright testing and automation library.
Deploy headless browsers in Docker. Run on our cloud or bring your own. Free for non-commercial uses.
Python APIs for web automation, testing, and bypassing bot-detection.
TranslatedWeekly Go Online Meetup via Bilibili｜Go Night Reading｜Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
webhook is a lightweight incoming webhook server to run shell commands