Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Repository Index
47,133 repositories indexed
1–24 of 38 · Most stars
TranslatedXiaohongshu notes | comment crawler, Douyin videos | comment crawler, Kuaishou videos | comment crawler, Bilibili videos | comment crawler, Weibo posts | comment crawler, Baidu Tieba posts | Baidu Tieba comment and reply crawler | Zhihu Q&A and articles | comment crawler
📨 The ultimate agentic social media scheduling tool 🤖
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
- wget
- browser-bookmarks
- pinboard
Python ProxyPool for web spider
TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Incredibly fast crawler designed for OSINT.
Low-code platform allows you to build business apps, enables you to quickly create internal tools such as dashboard, crud app, admin panel, crm, cms, etc. Supports PostgreSQL, MySQL, Supabase, GraphQL, MongoDB, MSSQL, Rest API, Hugging Face, Redis, etc. Automate workflows with schedule or webhook. Open source Retool.
- internal-tool
- internal-tools
- gui
- typescript
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
A graph node engine and editor written in Javascript similar to PD or UDK Blueprints, comes with its own editor in HTML5 Canvas2D. The engine can run client side or server side using Node. It allows to export graphs as JSONs to be included in applications independently.
- graph
- modular
- javascript
- visual-programming
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
TranslatedAn open-source AI agent for social media — discover trends, create content, publish everywhere, and learn what works across Xiaohongshu, Douyin, Zhihu, Bilibili, and more.🎨
- agent
- agent-skill
- content-automation
- content-creation
The open source and enterprise-ready webhooks service 🦀
- webhooks
- rust
- webhook-server
- webhook-service
Shuffle: A general purpose security automation platform. Our focus is on collaboration and resource sharing.
Socioboard is world's first and open source Social Technology Enabler. Socioboard Core is our flagship product.
- social-media
- social-network
Strategies to Gekko trading bot with backtests results and some useful tools.
- gekko
- strategies
- gekko-strategies
- gekko-backtest
Typed TypeScript SDK for direct and managed social platform integrations, with capability checks, explicit outcomes, and deterministic testing.
- api-integration
- bluesky
- bun
- developer-tools
.NET Implementation of the Slack team communication platform API.
The Sofie TV Studio Automation System, used in live TV news production by the Norwegian public service broadcaster NRK since September 2018
- sofie
- nrk
- sofie-server
- mos-protocol
TranslatedBullFlow — Open-source enterprise RAG solution: knowledge base → automated marketing content → omnichannel publishing. The entire knowledge base pipeline runs on local open-source models, with no paid models needed; AI Agent creates image-text posts / images / videos, with one-click publishing to 17 platforms including Douyin, Xiaohongshu, WeChat Official Accounts, LinkedIn, and YouTube. Commercial-grade, multi-tenant, billing & admin built in. MIT
- ai-agents
- bge-m3
- content-generation
- elasticsearch
🩵 Self-hosted & Fully-automated ActivityPub Bridge for Static Sites.
- activitypub
- federation
- fediverse
- rust
A full-featured radio automation system targeted for use in professional broadcast and media environments
Star the repo to help support! The open source operating system for creating viral content on social media.
🌍 Sentiment analysis over real-time social media data, rendering live charts to visualise trends
- sentiment-analysis
- sentiment-visualisation
- data-visualisation