🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Repository Index
47,133 repositories indexed
1–14 of 14 · Most stars
Crawl a site to generate knowledge files to create your own custom GPT from a URL
Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/
A framework to build Slack apps using Python
- slack
- slack-api
- bolt
- web-api
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
- xquery
- xml
- html
- json
Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Get video info from Dailymotion, Vimeo, Wistia, and YouTube URLs.
A Oembed consumer library, that gives you information about urls. It helps you replace urls to youtube or vimeo for example, with their html embed code. It has advanced features like offline support, responsive embeds and caching support.
Selfhosted. URL to PNG utility featuring parallel rendering using Playwright for screenshots and with storage caching via Local, S3, or CouchDB
Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
Extract URLs to stylesheets, scripts, links, images or HTML imports from HTML
Create HTML snippets/embeds from URLs using info from oEmbed, Open Graph, meta tags.
Gets events from an ical-URL/file, a caldav-server or from the iCloud.
- ical
- caldav
- events
- icloud