Sign in

Repository Index

47,133 repositories indexed

Tags: Automation, bots, and scraping
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–24 of 27 · Most stars

A 100% free modern JS SaaS boilerplate (React, NodeJS, Prisma). Full-featured: Auth (email, google, github, slack, MS), Email sending, Background jobs, Landing page, Payments (Stripe, Polar.sh), Shadcn UI, S3 file upload. AI-ready with tailored AGENTS.md, skills, and Claude Code plugin. One cmd deploy. Powered by Wasp full-stack framework.

  • full-stack
  • open-source
  • saas
  • nodejs

Python based web automation tool. Powerful and elegant.

  • automation-framework
  • python
  • web-automation

An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.

  • telegram
  • archive
  • wayback-machine
  • save-the-internet

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Extracts information about web pages, like youtube videos, twitter statuses or blog articles.

  • php
  • embed
  • oembed
  • opengraph

Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.

  • webkit
  • headless-browsers
  • erik
  • swift

Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.

Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.

  • crawling
  • csharp
  • elixir
  • ffi

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

  • go
  • golang
  • goquery
  • colly

Search the web and primary sources, read full pages, and check every citation. MCP server for Claude, Cursor, Codex and more. Runs on your machine.