Sign in

Repository Index

142,187 visible repositories in the index

Tags: All Tags
Tags

A repository shows when it has any selected Tag.

Filters

Filters

Clear all filters

25–48 of 627 · Most stars

An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.

The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.

Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9

Download pictures (or videos) along with their captions and other metadata from Instagram.

  • instagram
  • instagram-client
  • instagram-feed
  • downloader

Fast and reliable browser automation for e2e testing, scraping, and stealth. Bypasses bot-detection with CDP Mode. Has integrations for pytest, Playwright, and MCP.

TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.

  • webcrawler
  • scrapy
  • crawlab
  • spiders-management

Translated:orange_book: Chinese Xinhua Dictionary database. Includes xiehouyu (two-part allegorical sayings), idioms, words, and Chinese characters.

  • data
  • scraper
  • chinese-traditional
  • python3

jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

TranslatedAV movie management system, avmoo, javbus, javlibrary crawler, online AV movie library, AV magnet link database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database

  • javbus
  • avmoo
  • javlibrary
  • spider

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.