Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Repository Index
47,133 repositories indexed
1–24 of 47 · Most stars
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
- crawler
- crawling
- crawling-python
- playwright
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
- anti-detect
- bot-detection
- browser-automation
- chromium
Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
Zulip server and web application. Open-source team chat that helps teams stay productive and focused.
- zulip
- python
- python3
- chat
🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Command-line program to download image galleries and collections from several image hosting sites
- gallery
- downloader
- pixiv
- danbooru
Production-grade platform for building agentic IM bots - 生产级多平台智能机器人开发平台/ Agent、知识库编排、插件系统 / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信, 企微智能机器人, 公众号) / 飞书 / 钉钉 / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow
📷 Instagram Bot - Tool for automated Instagram interactions
- clarifai
- automation
- script
🎬 人人影视 机器人和网站，包含人人影视全部资源以及众多网友的网盘分享
- yyets
- telegram-bot
- movies
- tv-shows
Proxy server to bypass Cloudflare protection
- puppeteer
- cloudflare
- proxy
- chrome
holehe allows you to check if the mail is used on different sites like twitter, instagram and will retrieve information on sites with the forgotten password function.
- information-gathering
- osint
- osint-tools
- open-source-intelligence
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
Research and development (R&D) is crucial for the enhancement of industrial productivity, especially in the AI era, where the core aspects of R&D are mainly focused on data and models. We are committed to automating these high-value generic R&D processes through R&D-Agent, which lets AI drive data-driven AI. 🔗https://aka.ms/RD-Agent-Tech-Report
- agent
- ai
- automation
- data-mining
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
📊 APIs for Web Scraping, Browser Automation, and E2E Testing. CDP Mode & Stealthy Playwright Mode bypass bot-detection/CAPTCHAs.
- python
- selenium-python
- e2e-testing
- seleniumbase
Node-based Visual Programming Toolbox
- photogrammetry
- computer-vision
- 3d-reconstruction
- alicevision
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Multi-Agent Harness for Production AI
CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.
- parsers
- json
- linux
- command-line
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
- python
- stackstorm
- devops
- deployment
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready