📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
- articles
- articles-data
- crawler
- datasets-preparation
99,825 visible repositories in the index
49–72 of 73 · Most stars
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
Automated lending on Cryptocurrency exchanges Poloniex and Bitfinex
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
Curated list of free and low cost AI tools, LLM APIs, IDEs, agents, and infrastructure for building real AI apps
ARES - authorized red-team engagement automation with dashboard, campaign scope, module orchestration, OPSEC controls, encrypted vault, and reporting.
660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
htmlquery is golang XPath package for HTML query.
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
App-inspector is a mobile UI viewer in browser.
CorelDraw Desktop 2026 setup with professional color grading, HDR support, Fairlight audio post with unlimited tracks, and multi-user collaboration on a single timeline for Windows 11
An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
Run a web page like a program
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
querySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer
Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!
Self-hosted, invisible CAPTCHA that detects AI agents driving real browsers. GDPR, WCAG 2.2 AA, and EAA: no cookies, no third-party data sharing, and no visual or audio puzzle. With a false-positive budget enforced in CI. Proof of work, drop-in for Turnstile and reCAPTCHA.
Z-Hound — a CSV-first collector that finds public threat reports and extracts structured IoCs without a database, LLM, or API key.
:vertical_traffic_light: Block malicious crawlers with Cloudflare Firewall Rules
Visual Integration Designer for Apache Camel
Python/FastAPI application for monitoring Brazilian public-sector IT opportunities, with automated data collection, job filters, and official-source tracking.
Webpage summary extractor using Facebook Open Graph and arc90's readability
A node-red connector for remote instances of wa-automate-nodejs. Be sure to 🌟 this repository for updates!