Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Fast, reliable browser automation for web crawling, testing, scraping, and stealth. Bypasses CAPTCHAs with CDP Mode. Integrations include pytest, Playwright, MCP, and CLI.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
Fast, robust Google Flights scraper (API) for Python. (Probably)
- airport
- flights
- google-flights
- google-flights-api
muse.ai 注册与邀请码兑换协议研究(纯 HTTP Python 实现)
A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!
- chrome
- chrome-devtools-protocol
- python
- webscraping
Open-source framework for exporting your personal data.
Zotero extension that combats link rot by archiving webpages and journal articles.
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.