Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
70,234 visible repositories in the index
1–24 of 107 · Most stars
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A powerful JavaScript library for interacting with the Discord API
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app
- whatsapp-web
- whatsapp-bot
- whatsapp-api
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
- nlp
- machine-learning
- machine-learning-library
- bot
⬛️ GUI/CLI tool and library for saving complete web pages as a single HTML file
- save-the-internet
- procrastination
- its-mine
- no-more-404
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
- apify
- automation
- beautifulsoup
- crawler
Cross-platform library for building Telegram clients
- telegram
- cross-platform
- library
Go library for the WhatsApp web multidevice API
- go
- golang
- whatsapp-web
🔥 The fastest and powerful Python library for Instagram Private API 2026 with HikerAPI SaaS
- instagram-api
- instagram-client
- instagram-downloader
Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
- cdp
- chromium
- playwright
- puppeteer
An open source library for deep learning end-to-end dialog systems and chatbots.
An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more
Windows GUI Automation with Python (based on text properties)
- python
- automation
- ui-automation
- ui-automated-tests
A framework agnostic PHP library to build chat bots
- chatbot
- botman
- bot-framework
- chatbot-framework
📜 Extract meaningful content from the chaos of a web page
A Rust library for the Discord API.
- discord-api
- rust
- discord
- hacktoberfest
Java library to create bots using Telegram Bots API
- telegram-bot
- telegram-bots-api
- webhook
- java-library
Undetected version of the Playwright testing and automation library.
- automation
- bot
- bots
- botting
Jobs scraper library for LinkedIn, Indeed, Glassdoor, ZipRecruiter, Bayt, Naukri, BDJobs & HelloWork
- indeed
- ziprecruiter
- jobs-scraper
A library that scrapes Linkedin for user data
- linkedin-scraper
- scraper
- profile