Repository Index

Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots

• chat
• chat-application
• chatapp
• go

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

• webcrawler
• scrapy
• crawlab
• spiders-management

TranslatedA new-generation web crawler platform that defines crawler workflows graphically, so you can build a crawler without writing code.

• spider
• crawler
• jsoup
• xpath

Visual Development for React, Vue, Svelte, Qwik, and more

• react
• webcomponents
• wysiwyg
• frontend

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

• comic-downloader
• novel-downloader
• cejs
• downloader

新闻网页正文通用抽取器 Beta 版.

• python3
• webcrawler
• webspider

🚀 Curated collection of Amazing Python scripts from Basics to Advance with automation task scripts.

• projects
• python-projects
• machine-learning
• artificial-intelligence

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

• java
• webcrawling
• warc
• heritrix

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

• ai-agents
• cli
• llm
• markdown

TikTok LIVE API for Node.js: An unofficial library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.

• tiktok
• tiktok-api
• tiktok-live
• chat

蓝天采集器是一款开源免费的爬虫系统，仅需点选编辑规则即可采集数据，可运行在本地、虚拟主机或云服务器中，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

• crawler
• crawling
• spider
• webcrawler

Tinode web chat using React

• chat
• chat-application
• chatapp
• react

querySelector that can pierce Shadow DOM roots without knowing the path through nested shadow roots. Useful for automated testing of Web Components. Production use is not advised, this is for test environments/tools such as Web Driver, Playwright, Puppeteer

• webcomponents
• shadow-dom
• javascript
• puppeteer

Utility middleware for testing web components in AVA via Puppeteer.