Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.
Repository Index
47,133 repositories indexed
1–16 of 16 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Clone any website with one command using AI coding agents
An autonomous agent that conducts deep research on any data using any LLM providers
- ai
- python
- agent
- automation
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
- ai-agent
- anti-bot
- antidetect-browser
- automation
EveryInfra MCP server & agent plugin: scraper API for TikTok, Instagram, Amazon, YouTube, Xiaohongshu & Douyin. 573 data APIs across 106 platforms, from $0.56/1K calls. For Claude Code, Cursor, Codex.
- ai-agents
- claude-code
- cursor
- mcp
Self-hosted, invisible CAPTCHA that detects AI agents driving real browsers. GDPR, WCAG 2.2 AA, and EAA: no cookies, no third-party data sharing, and no visual or audio puzzle. With a false-positive budget enforced in CI. Proof of work, drop-in for Turnstile and reCAPTCHA.
- anti-bot
- behavioral-analysis
- bot-detection
- captcha
Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
- crawling
- csharp
- elixir
- ffi
Rust web scraper & crawler for AI agents: URLs, PDFs and Office files → clean Markdown/JSON. Firecrawl- and Jina Reader-compatible API, MCP server, Python/Node SDKs. Self-hosted single binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
The AI-native browser automation library. Snapshot + ref targeting — born from OpenClaw, built for agents, by agents.
Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.