Sign in

Repository Index

110,088 visible repositories in the index

Tags: Automation, bots, and scraping
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

337–360 of 668 · Most stars

Returns latest research results by crawling arxiv papers and summarizing abstracts. Helps you stay afloat with so many new papers everyday.

  • script
  • python
  • arxiv
  • summarization

Effortlessly extract data with our AI scraper API. Simplify data extraction, get clean JSON outputs and adapt to page changes. Try it free today!

  • ai-data-extraction
  • ai-scraper
  • ai-web-scraper
  • data-extraction

Calibre new douban metadata source plugin. Douban no longer provides book APIs to the public, so it can only use web crawling to obtain data. This is a calibre Douban plugin based on web crawling.

Go HTTP client with browser-identical TLS/HTTP2 fingerprinting. Bypass bot detection by perfectly mimicking Chrome, Firefox, and Safari at the cryptographic level (JA3/JA4, Akamai fingerprint, header order). Supports HTTP/1.1, HTTP/2, HTTP/3, sessions, cookies, and proxies.

  • anti-bot
  • bot-detection
  • browser-fingerprint
  • browser-fingerprinting

taskt (pronounced 'tasked' and formely sharpRPA) is free and open-source robotic process automation (rpa) built in C# powered by the .NET Framework

Visual Blocks for ML is a Google visual programming framework that lets you create ML pipelines in a no-code graph editor. You – and your users – can quickly prototype workflows by connecting drag-and-drop ML components, including models, user inputs, processors, and visualizations.

Vocat is an open-source web control panel and engineering toolkit for Quectel EC20/EC25-class cellular modems. It combines modem discovery, live radio status, AT and USSD terminals, SMS, WiFi Calling, eSIM management, network selection, proxy routing, notifications, audit logs, and release automation in one self-contained service.

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

  • python
  • lxml
  • xpath
  • xml

A high-performance Internet Relay Chat (IRCv3) server for UNIX-like and Windows systems

  • inspircd
  • c-plus-plus
  • irc-server
  • ircv3

An Azure Function solution to crawl through all of your image files in GitHub and losslessly compress them. This will make the file size go down, but leave the dimensions and quality untouched. Once it's done, ImgBot will open a pull request for you to review and merge. [email protected]

  • github-integration
  • azure-functions
  • dotnet-core
  • azure

Fast proxy scraper and checker written in Rust. Collects HTTP, SOCKS4 and SOCKS5 proxies from any number of sources, verifies each one really works, and writes JSON and plain text with response time, exit IP, ASN and geolocation. Single binary for Windows, Linux, macOS, Android, FreeBSD, NetBSD, illumos and Solaris, with an interactive TUI.

  • proxyscraper
  • proxyscrape
  • proxy-checker
  • proxy