AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.
The Tavily Python SDK allows for easy interaction with the Tavily API, offering the full range of our search, extract, crawl, map, and research functionalities directly from your Python programs. Easily integrate smart search, content extraction, and research capabilities into your applications, harnessing Tavily's powerful features.
Effortlessly extract data with our AI scraper API. Simplify data extraction, get clean JSON outputs and adapt to page changes. Try it free today!
- ai-data-extraction
- ai-scraper
- ai-web-scraper
- data-extraction
🤖 Scrape data from HTML websites automatically by just providing examples
Lightweight library for scraping web-sites with LLMs
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
Web data extraction tool implemented as chrome extension
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Just the facts -- web page content extraction
Extract urls from text
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
a tool to get Facebook data, and some Facebook bots, and extra tools found on Facebook Toolkit ++.
- facebook
- facebook-login
- facebook-graph-api
- facebook-api
Browser4 — an AI-native browser engine for autonomous agents, intelligent extraction, and large-scale web automation.
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
- articles
- articles-data
- crawler
- datasets-preparation
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
Crawly, a high-level web crawling & scraping framework for Elixir.
Free desktop SEO crawler - open source alternative to Screaming Frog and similar tools. Crawl websites, analyze links, extract SEO data, and export results without subscription fees. Fully customizable and extensible!
A Rust library to extract useful data from HTML documents, suitable for web scraping.
Universal content reader MCP Server for 10+ platforms
💬 Python scripts to parse Messenger, Hangouts, WhatsApp and Telegram chat logs into DataFrames.
- google-hangouts
- facebook-messenger
- wordcloud
- hangouts-logs
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
An AI assistant that lives in your browser. Built for collaboration, not autonomy theater. You guide, it executes. Automate repetitive web tasks, extract data from any website, and transform it into whatever you need.
ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions based on the content, or summarize the key points.
- chatgpt
- embedding
- gpt-35-turbo
- news-extractor