Repository Index

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

• ruby
• scraper
• crawler
• dsl

Telegram Client API (MTProto) library written 100% in C# and .NET

• telegram
• telegram-api
• dotnet
• csharp

:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine

• meta-search-engine
• rust
• search-engine
• fast

Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.

• java
• elasticsearch
• full-text-search
• lucene

A .NET library that converts cron expressions into human readable descriptions.

• cron
• cron-expression
• cron-libraries
• quartz

Tweetinvi, an intuitive Twitter C# library for the REST and Stream API. It supports .NET, .NETCore, UAP (Xamarin)...

• netcore
• portable
• twitter
• csharp

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

• xquery
• xml
• html
• json

LINQ Provider for the Twitter API (C# Twitter Library)

• twitter
• twitter-api
• twitter-library
• tweets

Functional HTML scraping and rewriting with CSS in OCaml

• scraping
• html
• soup
• ocaml

🐘 A PHP client for interacting with Gotenberg.

• gotenberg
• pdf
• word
• excel

TranslatedBullFlow — Open-source enterprise RAG solution: knowledge base → automated marketing content → omnichannel publishing. The entire knowledge base pipeline runs on local open-source models, with no paid models needed; AI Agent creates image-text posts / images / videos, with one-click publishing to 17 platforms including Douyin, Xiaohongshu, WeChat Official Accounts, LinkedIn, and YouTube. Commercial-grade, multi-tenant, billing & admin built in. MIT

• ai-agents
• bge-m3
• content-generation
• elasticsearch

Send Sir Perceval on a quest to retrieve and gather data from software repositories.

• grimoirelab
• perceval
• data-mining
• data-fetching

An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.

• elixir
• parser
• html
• xml

DocSearch - Scraper

• algolia
• docsearch
• documentation
• scraper

Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!

• browser
• browser-automation
• php
• php-browser

Template designer for automation

• network
• automation
• ansible
• jinja2

Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.

• crawling
• csharp
• elixir
• ffi

Z-Hound — a CSV-first collector that finds public threat reports and extracts structured IoCs without a database, LLM, or API key.

A robust web archive analytics toolkit

• python
• web
• warc
• bigdata

Human-readable schedule expressions for .NET (C#): write "every day at 9am except sunday" instead of cron. Time zones, fluent builder, custom keywords, cron converter.

• cron
• csharp
• date-time
• dotnet

An app to search startup jobs scraped from websites written in Elixir, Phoenix, React and styled-components.

• elixir
• phoenix
• otp
• react

Screenshot automation framework. Define once, regenerate forever. Visual picker, multi-viewport, light/dark mode, CI-ready.

• automation
• cli
• documentation
• playwright

api for liquipedia.net

• dota2
• liquipedia
• python3
• counter-strike

Uses parselets and rwget to generate csv files from websites