Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
Repository Index
47,133 repositories indexed
1–23 of 23 · Most stars
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust
- browser
- rust
- servo
- browser-automation
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
- golang
- chrome
- chrome-debugging-protocol
- testing
Incredibly fast crawler designed for OSINT.
TranslatedXiaohongshu (XiaoHongShu, RedNote) link extraction / post collection tool
Connect APIs, remarkably fast. Free for developers.
- eventsourcing
- event-sourcing
- automation
- workflow
Fast, scriptable CLI for the App Store Connect API. Automate TestFlight, builds, submissions, signing, analytics, screenshots, subscriptions, and more
- app-store-connect
- automation
- cicd
- cli
TranslatedThe Xianyu (闲鱼) Auto-Reply Management System is an automated customer service system developed with Python + FastAPI, designed specifically for the Xianyu platform. The system connects to Xianyu servers through WebSocket, receives and processes messages in real time, and provides intelligent automatic reply service. It also integrates features such as automatic Xianyu delivery, automatic reviews, and automatic polishing, enabling an automated workflow for virtual goods on Xianyu.
Blazing fast deployer and server automation tool
- ruby
- rails
- mina
- deployment
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
- crawler
- dotnetcore
- cross-platform
- csharp
ZapFast: a fast, native WhatsApp client in Rust and egui
- chat
- desktop-app
- egui
- linux
Fast website link checker in Go
A framework to build Slack apps using Python
- slack
- slack-api
- bolt
- web-api
:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine
- meta-search-engine
- rust
- search-engine
- fast
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
MargieBot is a .NET library designed to make building bots for Slack fast, easy, and fun.
Python/FastAPI application for monitoring Brazilian public-sector IT opportunities, with automated data collection, job filters, and official-source tracking.
Build beautiful Slack UIs fast
Lightweight and lightning fast Java WebDriver client for test automation professionals
SpiderMan,a base-on Broadway fast high-level web crawling & scraping framework for Elixir.