Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
Repository Index
78,351 visible repositories in the index
217–240 of 277 · Most stars
Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.
A Discord API wrapper library made in C
Open-source design agent: on-brand stills, decks, diagrams, charts and motion graphics from a chat thread, built on Deep Agents.
- ai-agent
- deepagents
- hyperframes
- langgraph
TranslatedBullFlow — Open-source enterprise RAG solution: knowledge base → automated marketing content → omnichannel publishing. The entire knowledge base pipeline runs on local open-source models, with no paid models needed; AI Agent creates image-text posts / images / videos, with one-click publishing to 17 platforms including Douyin, Xiaohongshu, WeChat Official Accounts, LinkedIn, and YouTube. Commercial-grade, multi-tenant, billing & admin built in. MIT
- ai-agents
- bge-m3
- content-generation
- elasticsearch
The unofficial HLTV Node.js API
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
Functional HTML scraping and rewriting with CSS in OCaml
Telegram Bot SDK for Swift (unofficial)
Telegram Bot API Wrapper written in Elixir
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
The unix-way web crawler
Scrape any website, article or RSS/Atom Feed with ease!
- scrape
- elixir
- readability
- rss
DocSearch - Scraper
- algolia
- docsearch
- documentation
- scraper
Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
- golang
- unmarshall
- goquery
- selectors
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
Playwright PHP library for browser automation: navigation, E2E tests, assertions, screenshots, and so much more!
- browser
- browser-automation
- php
- php-browser
Self-hosted, invisible CAPTCHA that detects AI agents driving real browsers. GDPR, WCAG 2.2 AA, and EAA: no cookies, no third-party data sharing, and no visual or audio puzzle. With a false-positive budget enforced in CI. Proof of work, drop-in for Turnstile and reCAPTCHA.
- anti-bot
- behavioral-analysis
- bot-detection
- captcha
Comprehensive wrapper and execution manager for the Chrome browser using the Chrome Debugging Protocol.
- chrome
- python
- chromium
- use-less-jerberscript
Turn any website into clean, structured data: Markdown, metadata and links from one page or a whole site, in every major language.
- crawling
- csharp
- elixir
- ffi
🔥 A daily streak system & portfolio for your projects.
- summer-of-making
- nextjs
- hackclub
- hacktoberfest
Search and read the web for AI agents at 99% fewer tokens. Free CLI + MCP.
MindAct: Community-Driven AI Agent for Claude, Obsidian, and RAG Knowledge Graphs
- activator
- analytics
- artificial-inteligence
- bigquery