Sign in

Repository Index

132,241 visible repositories in the index

Tags: Web scraper
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–24 of 70 · Most stars

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)

  • swift
  • swiftsoup
  • parse
  • html-document

Jobs scraper library for LinkedIn, Indeed, ZipRecruiter, Glassdoor, SEEK, Naukri, HelloWork, Bayt & BDJobs

  • indeed
  • linkedin
  • ziprecruiter
  • linkedin-scraper

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

  • python
  • bot
  • client
  • python3

An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices (e.g., jobs, scholarships); and sends a structured summary with a calculated match score directly to the assigned messenger number.

Shadow any website for offline viewing, with the JavaScript stripped out

AutoCLI is a Blazing fast, memory-safe command-line tool — Fetch information from any website with a single command. Covers Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and 55+ sites, with support for controlling Electron desktop apps, integrating local CLI tools (gh, docker, kubectl), now powered by AutoCLI.ai .

虎牙,斗鱼,抖音,BiliBili,TikTok,Twitch🔥热门🔥智能直播视频剪辑发布AI机器人,自动化🤖,全智能化⚙(智能生成切片,标题,封面,简介),可视化👓,平台热门监控🌡,丰富插件随意扩展🕹,快速部署⚡,视频账号打造自动发布🌟,支持DIY🎮