Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Repository Index
47,133 repositories indexed
1–24 of 28 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Scrapy, a fast high-level web crawling & scraping framework for Python.
- python
- scraping
- crawling
- framework
Automate the process of making money online.
The premier source of truth powering network automation. Open source under Apache 2. Try NetBox Cloud free: https://netboxlabs.com/products/free-netbox-cloud/
- dcim
- ipam
- netbox
- network-automation
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Command-line program to download image galleries and collections from several image hosting sites
- gallery
- downloader
- pixiv
- danbooru
TranslatedProduction-grade platform for building agentic IM bots - Agent, knowledge base orchestration, plugin system / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信 (Qiye Weixin, "Enterprise WeChat"), 企微智能机器人 (Qiwei Zhineng Jiqiren, "Enterprise WeChat smart bot"), 公众号 (Gongzhonghao, "official account")) / 飞书 (Feishu, "flying book") / 钉钉 (Dingding, "nail nail") / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
TranslatedWeChat assistant: 1. Sends customized messages to friends (a girlfriend) at scheduled times every day. 2. A bot automatically replies to friends. 3. Group assistant features (for example: looking up garbage sorting, weather, calendar, real-time movie box office, express delivery tracking, PM2.5, etc.)
- python
- python3
- itchat
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
TranslatedAutomatically claims the daily check-in points and all Growth Center rewards for WorkBuddy / Qoder CN — Buddy travel gifts, dispatching Buddy, opening blind boxes, claiming tasks, consecutive-login redemption, and make-up check-ins for broken login streaks are all covered. No extra dependencies to install; with one sentence, you can have AI automatically set up a Windows / macOS scheduled task, running silently with zero Token usage.
- auto-checkin
- automation
- python
- sign-in
Async Python 3.6+ web scraping micro-framework based on asyncio
- asyncio
- aiohttp
- asyncio-spider
- crawler
Chatbot in 200 lines of code using TensorLayer
- tensorlayer
- tensorflow
- chatbot
- rnn
:scissors: High performance, multi-threaded image scraper
WarcDB: Web crawl data as SQLite databases.
HAT (Hashcat Automation Tool) - An Automated Hashcat Tool for common wordlists and rules to speed up the process of cracking hashes during engagements. Created for Linux based systems
BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.
- python3
- data-science
- machine-learning
- python
Messages to slack from command line
- python
- slack
- command-line
MERK is an open source, cross-platform, multiple-document interface GUI IRC client for Windows, Linux, and macOS written in Python, PyQt5, and Twisted.
- python3
- pyqt5
- gplv3
- irc-client
:chart_with_upwards_trend: A live cryptocurrency historical trade data blotter. Download live historical trade data from any cryptoexchange, be it for machine learning, backtesting/visualizing trading strategies or for Quantopian/Zipline.
- cryptocurrency
- exchange
- trade
- data
Handles OAuth and stores slack token
Lightweight Docker monitoring dashboard with anomaly detection & Telegram alerts. One-liner install, zero config.