Open source file-system and knowledge based agent template. Build AI agents that stay up to date with your knowledge base
Repository Index
132,241 visible repositories in the index
145–168 of 248 · Most stars
A bot to help people with their rental real-estate search. 🏠🤖
- telegram-bot
- python
- mattermost-bot
- 2captcha
小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
- biquge
- qidian
- fiction
- novels
TweetScraper is a simple crawler/spider for Twitter Search without using API
- twitter-search
- scrapy
- tweets
The New (auto rotate) Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS :performing_arts:
- proxy
- proxy-server
- proxies
- proxy-list
Google play scraper for Python inspired by <facundoolano/google-play-scraper>
SiteOne Crawler is a cross-platform website crawler and analyzer for SEO, security, accessibility, and performance optimization—ideal for developers, DevOps, QA engineers, and consultants. Supports Windows, macOS, and Linux (x64 and arm64).
A scalable, mature and versatile web crawler based on Apache Storm
- web-crawler
- apache-storm
- distributed
- java
A tool for pixiv.net. 人人可用的P站爬虫
- pixiv
- crawler
- add-on
- batch
扫描“微信读书”已购图书并下载本地PDF的爬虫
- selenium
- weread
- web-crawler
- book-downloader
SpiderSuite (web security crawler) releases, wiki and roadmap
- cplusplus
- crawler
- osint-tool
- recon
Python website crawler.
Own the flow: one open-source AI platform for your knowledge, models and channels – web, WhatsApp, email, Outlook, Nextcloud & ownCloud. Local and cloud models side by side, with the cost on every answer.
A high performance web crawler / scraper in Elixir.
This is a scrapper to easily fetch any feed and interact with Instagram (like, follow, etc.) without OAuth for PHP.
- packagist
- php
- instagram-sdk
爬取百度百科中文页面,抽取三元组信息,构建中文知识图谱
爬虫逆向案例,已完成:TLS指纹|瑞数|震坤行 | 网易易盾 | 微信小程序反编译逆向(百达星系) | 同花顺 | rpc解密 | 加速乐 | 极验滑块验证码 | 巨量算数 | Boss直聘 | 企查查 | 中国五矿 | qq音乐 | 产业政策大数据平台 | 企知道 | 雪球网(acw_sc__v2) | 1688 | 七麦数据 | whggzy | 企名科技 | mohurd | 艺恩数据 | 欧科云链
Pure-Python protocol tools and research notes for X.com, with VibeLoft Twitter account datasets
High-performance asynchronous Douyin(抖音) TikTok Xiaohongshu(小红书) Kuaishou(快手) Weibo(微博) Instagram YouTube(油管) Twitter(X) Captcha Solver(验证码解决器) Temp Mail(临时邮箱) API(接口).
- douyin
- tiktok
- api
- captcha-solver
一个方便安全研究人员获取每日安全日报的爬虫和推送程序,目前爬取范围包括先知社区、安全客、Seebug Paper、跳跳糖、奇安信攻防社区、棱角社区以及绿盟、腾讯玄武、天融信、360等实验室博客,持续更新中。
A multi-thread crawler framework with many builtin image crawlers provided.
- crawler
- python
- scrapy
- google-images
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
Nuwax Agent OS - An enterprise-grade AI Agent Development and Operation Platform - Providing a complete solution for agent creation and distribution, knowledge base management, model proxy, memory system, and plugin ecosystem.
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
- sraping
- scrapper
- futures
- futures-historical-data