Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
Repository Index
109,091 visible repositories in the index
1–24 of 58 · Most stars
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
- web-scraping
- web-crawling
- npm
- headless-chrome
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
Turn any website into a structured API. Extract, automate, search and monitor the web.
- automation
- no-code
- scraper
- web-scraper
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.
Simple, powerful and flexible site generation framework with everything you love from Next.js.
- nextra
- nextjs
- static-site-generator
- mdx
OCR & Document Extraction using vision models
Zero-runtime Stylesheets-in-TypeScript
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Monorepo for tools developed by the Rush Stack community
An open source deep research clone. AI Agent that reasons large amounts of web data extracted with Firecrawl
Official Bright Data CLI - scrape, search, and extract structured web data directly from your terminal.
This extension adds extra autocompletion for laravel projects to VSCode.
- vscode
- vscode-extension
- laravel
- autocomplete
A pure javascript QR code reading library. This library takes in raw images and will locate, extract and parse any QR code found within.
- qrcode
- qr-parsing-library
- typescript
- qr
Extract any website’s design system into tokens in seconds: logo, colors, typography, borders & more. One command.
A React components library with Win95 UI
- react
- storybook
- windows95
- design-system
A CLI and library to extract information from React component files for documentation generation purposes.
- react
- documentation
- json
- docgen
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
- aitools
- crawl
- scrape
- serp
Your design system as an API. Connect AI to Figma for extraction, creation, and debugging.
- ai
- design-systems
- figma
- figma-api
Deobfuscate obfuscator.io, unminify and unpack bundled javascript
- ast
- bundle
- debundle
- extract
The swiss army chainsaw of terminal emulators
- terminal-emulators
- terminal
- typescript
- javascript
Production ready MCP server with real-time search, extract, map & crawl.
🎨 Extract prominent colors from an image
Magic Copy is a Chrome extension that uses Meta's Segment Anything Model to extract a foreground object from an image and copy it to the clipboard.
- chrome-extension
- computer-vision
- image-processing