47,133 repositories indexed
1–24 of 140 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Create agents that monitor and act on your behalf. Your agents are standing by!
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows. Used by 100,000+ builders.
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
Build a portrait of your life. Assistants that know you. Clones that talk like you.
Turn any website into a structured API. Extract, automate, search and monitor the web.
TranslatedA multi-task real-time/scheduled monitoring and intelligent analysis system for Xianyu (闲鱼), built with Playwright and AI, and equipped with a full-featured back-end management UI. It helps users find the products they like among the vast number of items on Xianyu.
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
A little like that j-thing, only in Go.
CLI for common Playwright actions. Record and generate Playwright code, inspect selectors and take screenshots.
Pythonic HTML Parsing for Humans™
Download pictures (or videos) along with their captions and other metadata from Instagram.
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
TranslatedWeChat assistant: 1. Sends customized messages to friends (a girlfriend) at scheduled times every day. 2. A bot automatically replies to friends. 3. Group assistant features (for example: looking up garbage sorting, weather, calendar, real-time movie box office, express delivery tracking, PM2.5, etc.)
A graph node engine and editor written in Javascript similar to PD or UDK Blueprints, comes with its own editor in HTML5 Canvas2D. The engine can run client side or server side using Node. It allows to export graphs as JSONs to be included in applications independently.
✨ The all-contributors bot website and documentation. Recognize all contributors, not just the ones who push code ✨
Real-time monitor and web admin for Celery distributed task queue
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.
Dark Web OSINT Tool
SwiftSoup: Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS)