TranslatedSelf-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Repository Index
78,351 visible repositories in the index
25–48 of 327 · Most stars
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.
Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.
🍃 Organic Maps is a free Android & iOS offline maps app for more than 12M travelers, tourists, hikers, and cyclists. It uses crowd-sourced OpenStreetMap data and is developed with love by the community. No ads, no tracking, no data collection, no crapware. Please donate to support the development!
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
- python
- news
- crawler
- crawling
Peach: Local-first video metadata scraper and media manager.
- local-first
- media-library
- python
- self-hosted
Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.
A cli tool to browse and play anime
Download pictures (or videos) along with their captions and other metadata from Instagram.
- instagram-client
- instagram-feed
- downloader
Pythonic HTML Parsing for Humans™
Next generation framework for web scraping, e2e testing, product tours, and bypassing bot-detection.
- python
- selenium-python
- e2e-testing
- seleniumbase
🦊 Anti-detect browser
- antidetect
- antidetect-browser
- fingerprint
- firefox
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier
- nsfw-classifier
- nsfw
- deep-learning
- content-moderation
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
- ai-agent
- anti-bot
- antidetect-browser
- automation
Translated:orange_book: Chinese Xinhua Dictionary database. Includes xiehouyu (two-part allegorical sayings), idioms, words, and Chinese characters.
- data
- scraper
- chinese-traditional
- python3
A scalable web crawler framework for Java.
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.
- perimeterx
- nopecha
- ai-agents
- anti-captcha
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
- browser-automation
- cdp
- golang
- headless-chrome