A Scala library for scraping content from HTML pages
Repository Index
142,273 visible repositories in the index
337–360 of 452 · Most stars
Twitter Internal API Document
- api
- unofficial
- documentation
Scrape Telegram group members (hidden members also) and add them to yours.
- telegram-group
- saver
- termux
- scraper
Two dumb distributed crawlers
- python3
- scrapy
- mongodb
- graphite
Telegram Members Adding Software/Script Using Termux.
- members-adder
- members-adding-script
- python
- telegram
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Fast, flexible, sync/async, Python 3.10+ screen scraping client specifically for network devices
Automated Tinder bot and scraper using selenium in python.
- tinder
- bot
- automation
- selenium
100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.
- ai-agent-tools
- ai-sdk
- better-auth
- cloudflare-workers
Scrape domain names from SSL certificates of arbitrary hosts
Firecrawl CLI and agent skills: search, scrape, and interact with the web from your terminal or any coding agent, plus parse, crawl, map, Agent and Monitor. Firecrawl is the context API for AI agents.
- agent-skills
- cli
- firecrawl
- mcp
Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.
A Facebook crawler
- scrapy
- crawler
- crawl
Python script, which empowers people with no programming background to generate robust leads on a mass scale. This repo will be compiled of various versatile techniques used in lead generation.
- lead-generation
- python
- scraper
- chromedriver
狠心开源企业级舆情新闻爬虫项目:支持任意数量爬虫一键运行、爬虫定时任务、爬虫批量删除;爬虫一键部署;爬虫监控可视化; 配置集群爬虫分配策略;👉 现成的docker一键部署文档已为大家踩坑 an enterprise-grade public opinion news crawler project: supports one-click operation of any number of crawlers, scheduled crawler tasks, batch deletion of crawlers; one-click deployment of crawlers
- apscheduler
- python
- scrapy
- scrapy-crawler
Declarative DOM extraction expression evaluator. 👨⚕️
Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.
- phantomjs
- python-support
- manga
- debian
Random User-Agent middleware based on fake-useragent
- web-scraping
- python
- scrapy
OSINT Swiss Army Knife
- go
- golang
- osint
- haveibeenpwnd
Twitter data scraping, embedding based image search and more.
OSINT tool - gets data from services like shodan, censys etc. in one app
Kemono pawchive Downloader is a fast, PyQt5 app for archiving content from a wide array of sites, including Kemono, Coomer, Bunkr, Erome, Saint2.su, nhentai, and Discord. It features a Creator Browser, update checker, and supports multi-threaded, multi-part downloads. Sessions can pause, resume, or recover. filters
- coomer-downloader
- coomer-su
- kemono
- kemono-downloader
Twitter Scraper
- twitter-api
- python
- tweets
Comprehensive guide for NetworkChuck Episode 480 - Robin AI Dark Web Scraping Tool. Installation, usage, safety guidelines, and troubleshooting for educational security research.