Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Repository Index
55,140 visible repositories in the index
49–72 of 87 · Most stars
WarcDB: Web crawl data as SQLite databases.
:computer: Facebook command line client (needs to be updated)
- facebook-api
- command-line
Scrape any website, article or RSS/Atom Feed with ease!
- scrape
- elixir
- readability
- rss
Send Sir Perceval on a quest to retrieve and gather data from software repositories.
- grimoirelab
- perceval
- data-mining
- data-fetching
An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
🌍 Sentiment analysis over real-time social media data, rendering live charts to visualise trends
- sentiment-analysis
- sentiment-visualisation
- data-visualisation
Z-Hound — a CSV-first collector that finds public threat reports and extracts structured IoCs without a database, LLM, or API key.
Self-hosted, invisible CAPTCHA that detects AI agents driving real browsers. GDPR, WCAG 2.2 AA, and EAA: no cookies, no third-party data sharing, and no visual or audio puzzle. With a false-positive budget enforced in CI. Proof of work, drop-in for Turnstile and reCAPTCHA.
- anti-bot
- behavioral-analysis
- bot-detection
- captcha
Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
- crawling
- csharp
- elixir
- ffi
Human-readable schedule expressions for .NET (C#): write "every day at 9am except sunday" instead of cron. Time zones, fluent builder, custom keywords, cron converter.
- cron
- csharp
- date-time
- dotnet
🐦💬 Update a gist to contain your latest tweet
- pinned-gist
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Translated🎮 Update profile README / pinned gist to contain your Steam playtime leaderboard.
- steam
- steam-playtime-leaderboard
- pinned-gist
- profile
Translated📊 Update profile README / pinned gist to contain your weekly WakaTime stats. This is a Golang implementation, see the original version https://github.com/matchai/waka-box. Display your weekly WakaTime language usage statistics in your profile README / pinned gist.
- wakatime-api
- wakatime
- pinned-gist
- weekly-wakatime-stats
The Frank!Framework is an easy-to-use, stateless integration framework which allows (transactional) messages to be modified and exchanged between different systems.
- java
- system-integration
- open-source
- community-driven
📌 Update a gist to contain your GitHub stats
- gist
- github-actions
- github
- pinned-gist
A robust web archive analytics toolkit
- python
- web
- warc
- bigdata
Upload and manage your assets in the iTunes Store using the iTunes Store’s Transporter (iTMSTransporter).
- ruby
- itunes
- app-store
- movies
Python/FastAPI application for monitoring Brazilian public-sector IT opportunities, with automated data collection, job filters, and official-source tracking.
:alarm_clock: Parse Cron Expressions, Compose Cron Expression Strings and Caluclate Execution Dates.
- elixir
- crontab
- cron
- schedule
Credential Phish Analysis and Automation
Docker service for automatically updating Docker swarm services whenever their image is updated.
- docker
- docker-swarm
- self-hosted
- automation