Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–15 of 15 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Create agents that monitor and act on your behalf. Your agents are standing by!
- automation
- notifications
- scraper
- webscraping
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
- anti-detect
- bot-detection
- browser-automation
- chromium
Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
An autonomous agent that conducts deep research on any data using any LLM providers
- ai
- python
- agent
- automation
Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
A cli tool to browse and play anime
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
- datadome
- perimeterx
- akamai
- antidetect-browser
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
- xquery
- xml
- html
- json
Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
- rvest
- rselenium
- httr
- scrape-websites
Chemical Information from the Web
- rstats
- ropensci
- chemical-information
- webscraping
:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Physical Bitcoin Stat Ticker
Browser extension to simulate browsing behaviour in search engines.
- automation
- audit
- browser-extension
- firefox