55,140 visible repositories in the index
25–48 of 87 · Most stars
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
The AI local media metadata crawler
🤖 Keep your forks up-to-date via automated PRs
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Turn any webpage into structured data using LLMs
Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Build databases, automations, apps & agents with AI — no code. Open source platform available on cloud and self-hosted. GDPR, HIPAA, SOC 2 compliant. Best Airtable alternative.
Official Bright Data CLI - scrape, search, and extract structured web data directly from your terminal.
Free, open-source crypto trading bot, automated bitcoin / cryptocurrency trading software, algorithmic trading bots. Visually design your crypto trading bot, leveraging an integrated charting system, data-mining, backtesting, paper trading, and multi-server crypto bot deployments.
CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
Fully functional Twitter clone built in flutter framework using Firebase realtime database and storage
File Attachment toolkit for Ruby applications
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
CLI tool to automate docker image updates. Interactive or unattended with notifications, image backups, autoprune, no pre-pulling and more.
The fastest way to understand code
Strategies to Gekko trading bot with backtests results and some useful tools.
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of User Interface or API, also known as Headless browser. It can be used to run automated tests / snapshots and manipulate websites using Javascript.
Crawly, a high-level web crawling & scraping framework for Elixir.
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Extract structured data from web sites. Web sites scraping.