使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
Repository Index
78,850 visible repositories in the index
241–264 of 707 · Most stars
Scrapy+Splash for JavaScript integration
- scrapy
- headless-browsers
TranslatedMulti-platform content monitoring, collection, and reposting: one web panel to manage Douyin / Xiaohongshu / Kuaishou / WeChat Channels / TikTok
- automation
- content-monitoring
- crawler
- douyin
Visual scripting framework for python
- python
- pyflow
- dataflow-programming
- qt
AI Powered Knowledge Graph Generator
- artificial-intelligence
- knowledge-distillation
- knowledge-graph
- llm
A OpenMMLAB toolbox for human pose estimation, skeleton-based action recognition, and action synthesis.
- action-recognition
- deep-learning
- graph-convolutional-network
- pytorch
Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)
- graph-attention-networks
- attention-mechanism
- self-attention
- pytorch
FlameScope is a visualization tool for exploring different time ranges as Flame Graphs.
A service daemon to run Scrapy spiders
Universal memory layer for AI Agents. It provides scalable, extensible, and interoperable memory storage and retrieval to streamline AI agent state management for next-generation autonomous systems.
- ai
- memory
- memory-management
- python
StellarGraph - Machine Learning on Graphs
- graphs
- machine-learning
- machine-learning-algorithms
- graph-convolutional-networks
[ICLR 2024] Official implementation of DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior
- 3d-generation
- aigc
- diffusion-models
- generative-model
A simple tool to update bib entries with their official information (e.g., DBLP or the ACL anthology).
- bibtex
- latex
- publication
- research-paper
Make your job hunt easy by automating your application process with this Auto Applier
- python
- selenium
- webscraping
- automation
Lightweight tool to automate downloading and metadata generation with yt-dlp
A python program that turns an LLM, running on Ollama, into an automated researcher, which will with a single query determine focus areas to investigate, do websearches and scrape content from various relevant websites and do research for you all on its own! And more, not limited to but including saving the findings for you!
A Pythonic wrapper for the Wikipedia API
Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
- yaml
- jq
- command-line
- json
Linux daemon for Intel CPU power limits and firmware-induced throttling.
- linux
- cpu-throttling
- intel
- intel-cpu
MyPaint is a simple drawing and painting program that works well with Wacom-style graphics tablets.
- painting
- drawing
- art
- mypaint
PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models (CIKM 2021)
- pytorch
- graph-neural-networks
- temporal-networks
- temporal-graphs
Tweets metadata scraper & activity analyzer
Token-scale scientific discovery engine — autonomous hypothesis generation, experiment execution, and knowledge graph synthesis
A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.