Sign in

Repository Index

108,094 visible repositories in the index

Tags: Automation, bots, and scraping
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

97–120 of 659 · Most stars

TranslatedZotero translators for Chinese web page scraping 🎉 This is Zotero translators for Chinese Sites(beta), not the official Zotero repo

  • zotero
  • translators
  • javascript
  • cnki

Swiss-army tool for scraping and extracting data from online assets, made for hackers

  • css
  • curl
  • gjson
  • json

Jobs scraper library for LinkedIn, Indeed, Glassdoor, ZipRecruiter, Bayt, Naukri, BDJobs, HelloWork & SEEK

  • indeed
  • linkedin
  • ziprecruiter
  • linkedin-scraper

An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices (e.g., jobs, scholarships); and sends a structured summary with a calculated match score directly to the assigned messenger number.

Official Repository: Telegram bot which can download direct links, torrents, nzb, google drive, telegram document, any file/folder from rclone supported clouds, all yt-dlp supported sites and jdownloader supported sites, then upload them to google drive, telegram cloud or to one of rclone supported clouds

  • mirror
  • leech
  • clone
  • mega

GPP is Android's unofficial release automation Gradle Plugin. It can do anything from building, uploading, and then promoting your App Bundle or APK to publishing app listings and other metadata.

  • android-development
  • play-store
  • gradle-plugin
  • gpp

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

  • crawler
  • dotnetcore
  • cross-platform
  • csharp

Automatically crawls proxy nodes on the public internet, de-duplicates and tests for usability and then provides a list of nodes

Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language Models in this new AI era

  • crawler
  • python3
  • scylla
  • proxy-pool

Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers

  • scanner
  • crawler
  • information-gathering
  • admin-scanner

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

  • endpoints
  • endpoint-discovery
  • bugbounty
  • crawler