WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
Repository Index
132,241 visible repositories in the index
289–312 of 1,135 · Most stars
All In One Web Recon
- python3
- pentesting
- pentest-tool
- pentesting-tools
AI Search & RAG Without Moving Your Data. Get instant answers from your company's knowledge across 100+ apps while keeping data secure. Deploy in minutes, not months.
- search
- search-engine
- federated-query
- federated-search
A powerful browser crawler for web vulnerability scanners
- headless-chrome
- crawler
- chrome-devtools
- golang
Open-source desktop AI agent for tools, files, knowledge, workflows, and real deliverables.
Gospider - Fast web spider written in Go
Node.js scraper to get data from Google Play
A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.
AI-powered cross-platform e-book reader with semantic search, RAG chat, local vector store, notes, TTS, and WebDAV sync.
- ai
- anthropic
- ebook-reader
- embeddings
pingcap/autoflow is a Graph RAG based and conversational knowledge base tool built with TiDB Serverless Vector Storage. Demo: https://tidb.ai
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
- scraper
- twitter-api
- twitter-bot
[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
- artificial-intelligence
- efficient
- knowledge-editing
- large-language-models
The open-source ontology runtime for AI agents.
- open-source
- ontology
- rdf
- knowledge-graph
EverRoom - A workspace that remembers your projects, decisions, and sources.
Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
- ai-native
- description-logics
- knowledge-graph
- mcp
Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.
- google-trend
- google-trend-api
- google-trend-strategy
Markdown powered Knowledgebase Wiki for Node.js
owllook-小说搜索引擎
A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
- search-engine
- python
- scraping
- crawler
AI-first knowledge base and startup OS
Bookmark manager for the wizards 🧙
- bookmarks
- self-hosted
- typescript
- docker
Your Self-Hosted Knowledge Base
- docker
- evernote
- free
- hosted
"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)
- controlnet
- knowledge-distillation
- pose-estimation
- stable-diffusion-webui
:octocat::fire: :star2::star::star::star: :star: 史上最全的BAT大厂Android面试题汇集,以及常用的Android开发的一些技能点,冷门知识点汇总,开发中遇到的坑汇总等干货。