TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
97,836 visible repositories in the index
1–24 of 38 · Most stars
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
TranslatedWeekly Go Online Meetup via Bilibili|Go Night Reading|Share Go-related technical topics through live streaming on Bilibili, and every day everyone communicates and discusses programming topics in a timely way on WeChat/Telegram/Slack.
bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
Pholcus is a distributed high-concurrency crawler software written in pure golang
A flexible event/agent & automation system with lots of bees 🐝
TranslatedAn API framework with a modular design based on Gin, which wraps commonly used features, is simple to use, and is dedicated to fast business development. For example, it supports cors cross-origin requests, jwt signature verification, zap log collection, panic exception capture, trace link tracing, prometheus monitoring metrics, swagger document generation, viper configuration file parsing, the gorm database component, the gormgen code generation tool, the graphql query language, errno unified definition of error codes, the use of gRPC, cron scheduled tasks, and so on.
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
Automatically crawls proxy nodes on the public internet, de-duplicates and tests for usability and then provides a list of nodes
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
面向 AI Agent 的微信公众号创作与发布 CLI:Markdown 排版、AI 配图、预览与草稿创建;支持由浏览器 Agent 保存知乎、CSDN、头条未发布草稿。
Shadow any website for offline viewing, with the JavaScript stripped out
一款快速、全面、易用的页面信息提取工具,可快速发现和提取页面中的JS、URL和敏感信息。
✨ A complete and self-contained solution for developers of any qualification to create a production-ready project with backend (Go), frontend (JavaScript, TypeScript) and deploy automation (Ansible, Docker) by running only one CLI command.
多播放源自动采集在线影视网站 golang vue
Unofficial open source mtproto server written in golang with compatible telegram client.
For detecting anti-detect browsers and spoofed profiles through runtime coherence analysis.
A telegram bot that will give instant stream links for telegram files without the need of waiting till the download completes. (Telegram File Stream Bot)
supports Telegram, Discord, Slack, Lark(飞书),钉钉, 企业微信, QQ, 微信, compatible with various LLMs including OpenAI, Gemini, DeepSeek, Doubao, and OpenRouter. It offers intelligent conversation, image generation, video creation, and more. Works seamlessly in both private chats and group settings.
open source grok bot. gawk bots automate your menial work via AI models and build you microapps to manage the outcome, so that you have a false sense of control.
Spoof TLS/JA3 fingerprints in GO and Javascript
🦋 Full-native go implementation of Telegram API
Port of Wappalyzer (uncovers technologies used on websites) to automate mass scanning.
A golang implementation of a console-based trading bot for cryptocurrency exchanges
Android automation from code. Test your apps on real phones, script the tasks you repeat, run a shelf of phones, or let an AI agent use one, from Python, Node.js, the CLI, HTTP or Appium clients. No ADB, no USB cable, no root.