Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–24 of 149 · Most stars
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
- agent-infrastructure
- ai-agent
- ai-search
- automation
Your lightweight open-source personal agent.
🧡 Everything is RSSible
TranslatedA visual no-code/code-free web crawler/spider. 易采集 (EasySpider, "easy collection"): a visual browser automation testing/data collection/web crawler software that lets you design and execute crawler tasks graphically without code. Alias: ServiceWrapper, an intelligent service wrapping system for Web applications.
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
AI agents, automations and apps that run your operations. Model agnostic.
- no-code-platform
- crud-app
- crud-application
- data-application
A powerful JavaScript library for interacting with the Discord API
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app
- whatsapp-web
- whatsapp-bot
- whatsapp-api
Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust
- browser
- rust
- servo
- browser-automation
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
- nlp
- machine-learning
- machine-learning-library
- bot
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
TranslatedAutomatically upload videos to social media: Douyin, Xiaohongshu, Shipinhao (WeChat Channels), TikTok, YouTube, Bilibili
- bilibili
- douyin
- tiktok
- xiaohongshu
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Python version of the Playwright testing and automation library.
- playwright
- chromium
- firefox
- webkit
Incredibly fast crawler designed for OSINT.
Browser Automation, End-to-End Testing, and Stealth in one library. Bypass bot-detection/CAPTCHAs with SeleniumBase CDP Mode.
- python
- selenium-python
- e2e-testing
- seleniumbase
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
TranslatedWeekly Go Online Meetup via Bilibili｜Go Night Reading｜Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
- golang
- gopher
- shenzhen
- offline
A standalone version of the readability lib
TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
- javbus
- avmoo
- javlibrary
- spider
✨ The all-contributors bot website and documentation. Recognize all contributors, not just the ones who push code ✨
- contributors
- all-contributors
- opensource
- opensource-management