Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–24 of 109 · Most stars
Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
JavaScript API for Chrome and Firefox
- headless-chrome
- testing
- web
- developer-tools
all of the workflows of n8n i could find (also from the site itself)
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
- anti-detect
- bot-detection
- browser-automation
- chromium
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
Crawl a site to generate knowledge files to create your own custom GPT from a URL
A browser extension for automating your browser by connecting blocks
- vue
- chrome-extension
- automation
- browser-extension
Command-line program to download image galleries and collections from several image hosting sites
- gallery
- downloader
- pixiv
- danbooru
Bridge local AI coding agents (Claude Code, Cursor, Gemini CLI, Codex) to messaging platforms (Feishu/Lark, DingTalk, Slack, Telegram, Discord, LINE, WeChat Work). Chat with your AI dev assistant from anywhere — no public IP required for most platforms.
Proxy server to bypass Cloudflare protection
- puppeteer
- cloudflare
- proxy
- chrome
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit
Deploy headless browsers in Docker. Run on our cloud or bring your own. Free for non-commercial uses.
C# application with primary purpose of farming Steam cards from multiple accounts simultaneously.
- steam
- steam-api
- steam-account
- steambot
Download pictures (or videos) along with their captions and other metadata from Instagram.
- instagram-client
- instagram-feed
- downloader
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
- golang
- chrome
- chrome-debugging-protocol
- testing
Browser Automation, End-to-End Testing, and Stealth in one library. Bypass bot-detection/CAPTCHAs with SeleniumBase CDP Mode.
- python
- selenium-python
- e2e-testing
- seleniumbase
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Captcha solver extension for humans, available for Chrome, Edge and Firefox
- browser-extension
- chrome-extension
- firefox-extension
- captcha
Lighter web automation with Python
- selenium
- selenium-python
- python
- python3