AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
105,102 visible repositories in the index
25–48 of 153 · Most stars
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
Utilities for downloading the EPG (Electronic Program Guide) for thousands of TV channels from hundreds of sources.
Turn any website into an API your agent can call. Learns from the site's own frontend, heals itself when the site changes.
xlwings is a Python library that makes it easy to call Python from Excel and vice versa. It works with Excel on Windows, macOS, and Excel on the web.
Use Google News Scraper API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.
The simplest desktop client for OpenClaw 🦞 — bridge your Agent to WeChat, Feishu, Slack & Discord in one click. Works with Claude Code, Codex & any LLM. BYOK, Oauth, local-first, chat from your phone 24/7.
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos, contact information, followers, following and much more.
A metasearch library that aggregates results from diverse web search services
An Implementation of NTQQ Protocol, with Pure C#, Derived from Konata.Core
AutoCLI is a Blazing fast, memory-safe command-line tool — Fetch information from any website with a single command. Covers Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and 55+ sites, with support for controlling Electron desktop apps, integrating local CLI tools (gh, docker, kubectl), now powered by AutoCLI.ai .
Perplexity Scraper Track brand mentions, analyze rankings, and gain competitor intelligence from Perplexity. Get started in minutes.
Node.js scraper to get data from Google Play
Programmatically collect normalized news from (almost) any website.
JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.
An extensible message tunneling chat bot framework. Delivers messages to and from multiple platforms and remotely control your accounts.
Bridge Claude Code / Codex to IM platforms — chat with AI coding agents from Telegram, Discord, or Feishu/Lark.
A powerful Telegram bot that provides remote access to Claude Code, enabling developers to interact with their projects from anywhere with full AI assistance and session persistence.
Export tweets, bookmarks, lists and much more from Twitter(X) web app. (推文/书签/收藏/列表导出工具)
Control Google Chrome from the command line
Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal, GhostArchive & Intelligence X!
Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.
Generate and download e-books from online sources.
Download images from Pixiv and more!