🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automation, Home Assistant, and Networking.
Crawl a site to generate knowledge files to create your own custom GPT from a URL
A 100% free modern JS SaaS boilerplate (React, NodeJS, Prisma). Full-featured: Auth (email, google, github, slack, MS), Email sending, Background jobs, Landing page, Payments (Stripe, Polar.sh), Shadcn UI, S3 file upload. AI-ready with tailored AGENTS.md, skills, and Claude Code plugin. One cmd deploy. Powered by Wasp full-stack framework.
- full-stack
- open-source
- saas
- nodejs
Saves Discord chat logs to a file
File Attachment toolkit for Ruby applications
- file-upload
- ruby
- rack
- storage
An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services including Internet Archive, archive.today, Ghostarchive, IPFS, Telegraph, and file systems.
- telegram
- archive
- wayback-machine
- save-the-internet
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.
- java
- elasticsearch
- full-text-search
- lucene
A high performance web crawler / scraper in Elixir.
Appium Server setup to automate android testing on real devices
Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
A simple yet highly configurable Mastodon bot that posts geotagged aerial imagery of a random location in the world.
Translated🎮 Update profile README / pinned gist to contain your Steam playtime leaderboard.
- steam
- steam-playtime-leaderboard
- pinned-gist
- profile
Translated📊 Update profile README / pinned gist to contain your weekly WakaTime stats. This is a Golang implementation, see the original version https://github.com/matchai/waka-box. Display your weekly WakaTime language usage statistics in your profile README / pinned gist.
- wakatime-api
- wakatime
- pinned-gist
- weekly-wakatime-stats
Uses parselets and rwget to generate csv files from websites
Gets events from an ical-URL/file, a caldav-server or from the iCloud.
- ical
- caldav
- events
- icloud
Showcase your Telegram channels, groups, and bots with beautiful, dynamic preview cards. Perfect for GitHub profiles and portfolios. ✨
- developer-tool
- nextjs
- profile-badge
- profile-readme
atwSCADA — Free SCADA Software in a Single HTML File | IEC 61850, OPC UA, Modbus TCP | 53 Function Blocks, 65 Graphic Elements | No Installation | Download Free Demo
- digital-substation
- hmi
- iec61850
- industrial-automation
A package to fetching the maker profile - Super Mario Maker
- json
- mario-maker
- nodejs
- javascript