Python scraper based on AI
- scraping
- scraping-python
- llm
- web-crawler
71,372 visible repositories in the index
1–17 of 17 · Most stars
Python scraper based on AI
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
TranslatedProduction-grade platform for building agentic IM bots - Agent, knowledge base orchestration, plugin system / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信 (Qiye Weixin, "Enterprise WeChat"), 企微智能机器人 (Qiwei Zhineng Jiqiren, "Enterprise WeChat smart bot"), 公众号 (Gongzhonghao, "official account")) / 飞书 (Feishu, "flying book") / 钉钉 (Dingding, "nail nail") / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow
ChatterBot is a machine learning, conversational dialog engine for creating chat bots
Build voice agents with open-source models
Translated:orange_book: Chinese Xinhua Dictionary database. Includes xiehouyu (two-part allegorical sayings), idioms, words, and Chinese characters.
An Open-Source Asynchronous Coding Agent
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
TranslatedAI Vtuber is a virtual streamer [Live2D/UE/xuniren] powered by [ChatterBot/ChatGPT/claude/langchain/chatglm/text-gen-webui/闻达 (Wenda)/千问 (Qianwen)/kimi/ollama]. It can interact with viewers in real time in live streams on [Bilibili/抖音 (Douyin)/快手 (Kuaishou)/微信视频号 (WeChat Channels)/拼多多 (Pinduoduo)/斗鱼 (Douyu)/YouTube/twitch/TikTok], or chat directly on the local machine. It uses TTS technology [edge-tts/VITS/elevenlabs/bark/bert-vits2/睿声 (Ruisheng)] to generate replies and can optionally change the voice with [so-vits-svc/DDSP-SVC]; commands work together with SD to draw images.
Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language Models in this new AI era
Interact with your SQL database, Natural Language to SQL using LLMs
Advanced Multi-Featured Telegram UserBot, Built in Python Using Telethon lib.
🥂 Gracefully face hCaptcha challenge with multimodal large language model.
Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis
💬 RasaGPT is the first headless LLM chatbot platform built on top of Rasa and Langchain. Built w/ Rasa, FastAPI, Langchain, LlamaIndex, SQLModel, pgvector, ngrok, telegram
Library to scrape and clean web pages to create massive datasets.