newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
132,241 visible repositories in the index
25–48 of 376 · Most stars
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Graph-Native Semantic Layer for Trustworthy Enterprise AI
Incredibly fast crawler designed for OSINT.
TranslatedDistill high-value content from books, long-form videos, podcasts, and more into executable Agent Skills
Pocket Flow: Codebase to Tutorial
BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.
GEO-first SEO skill for Claude Code. Comprehensive AI search optimization for any website — citability scoring, AI crawler analysis, brand authority, schema markup, platform-specific optimization, and PDF reports. If you want learn how to sell this to real businesses, check out the skool community
Open-source intelligence for the global theater. Track everything from the corporate/private jets of the wealthy, and spy satellites, to seismic events in one unified interface. Hook an AI agent up to have it parse through data and find previously unseen correlations. The knowledge is available to all but rarely aggregated in the open, until now.
TranslatedPython scripts. Simulated login to Zhihu, web crawlers, working with Excel, WeChat official accounts, remote power-on
DeFi Developer roadmap is a curated Developer handbook which includes a list of the best tools for DApps development, resources and references!
TranslatedOneForAll is a powerful subdomain collection tool
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
KAG is a logical form-guided reasoning and retrieval framework based on OpenSPG engine and LLMs. It is used to build logical reasoning and factual Q&A solutions for professional domain knowledge bases. It can effectively overcome the shortcomings of the traditional RAG vector similarity calculation model.
TranslatedA self-media customer acquisition agent tailor-made for OPC/small, medium and micro enterprises
TranslatedA knowledge graph for mathematics (primary school to high school), English (grade 3 to high school), and junior high school physics: 1,644 knowledge points, 2,403 prerequisite edges, and 6,723 questions, aligned with the curriculum standards, with an adaptive prerequisite diagnostic.
TranslatedPython API for JMComic | Provides a Python API for accessing 禁漫天堂 (JMComic), supporting both the web and mobile clients | 禁漫天堂 (JMComic) GitHub Actions downloader🚀
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
TranslatedSelf-hosted knowledge agent platform for RAG, knowledge graphs and multi-agent workflows. Yuxi = Cloud Agents + Knowledge RAG, Self-hosted knowledge agent platform for RAG, knowledge graphs and multi-agent workflows.
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
The AI local media metadata crawler
TranslatedA tutorial and implement of disease centered Medical knowledge graph and qa system based on it. Knowledge graph construction, automatic question answering, KG-based automatic question answering. A medium-scale medical-domain knowledge graph centered on diseases, which is used to provide automatic question answering and analysis services.
🔥 The fastest and powerful Python library for Instagram Private API 2026 with HikerAPI SaaS
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML