Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–24 of 99 · Most stars
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
🧡 Everything is RSSible
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
A powerful JavaScript library for interacting with the Discord API
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app
- whatsapp-web
- whatsapp-bot
- whatsapp-api
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
- nlp
- machine-learning
- machine-learning-library
- bot
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
TranslatedAutomatically upload videos to social media: Douyin, Xiaohongshu, Shipinhao (WeChat Channels), TikTok, YouTube, Bilibili
- bilibili
- douyin
- tiktok
- xiaohongshu
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit
TranslatedWeekly Go Online Meetup via Bilibili｜Go Night Reading｜Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
- golang
- gopher
- shenzhen
- offline
A standalone version of the readability lib
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more
A framework agnostic PHP library to build chat bots
- chatbot
- botman
- bot-framework
- chatbot-framework
A Rust library for the Discord API.
- discord-api
- rust
- discord
- hacktoberfest
Mechanize is a ruby library that makes automated web interaction easy.
- scraping
- web
- ruby
TranslatedAn open-source AI agent for social media — discover trends, create content, publish everywhere, and learn what works across Xiaohongshu, Douyin, Zhihu, Bilibili, and more.🎨
- agent
- agent-skill
- content-automation
- content-creation
A library to support tests with throwaway instances of Docker containers for all compatible .NET Standard versions.
- automation
- dotnet
- docker
- testcontainers
The most popular PHP library for use with the Twitter OAuth REST API.
- twitter-oauth
- php
- twitter-api
- php-library
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
Headless chrome/chromium automation library (unofficial port of puppeteer)
- automation
- puppeteer
- chromium
A node.js library for testing modern web applications
- testing-tools
- headless
- headless-browsers
- headless-chrome