Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Repository Index
47,133 repositories indexed
1–8 of 8 · Most stars
🛏 An HTML to Markdown converter written in JavaScript
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- web-scraping
- text-extraction
- nlp
- html2text
⚙️ Convert HTML to Markdown. Even works with entire websites and can be extended through rules.
- go
- golang
- html
- html-to-markdown
CommonMark/Markdown Java parser with source level AST. CommonMark 0.28, emulation of: pegdown, kramdown, markdown.pl, MultiMarkdown. With HTML to MD, MD to PDF, MD to DOCX conversion modules.
- commonmark
- pegdown
- java
- markdown-parser
Convert HTML to Markdown with PHP
Turn messy, real-world HTML into clean Markdown, from every major language.
- html-converter
- markdown-converter
- rag
- text-extraction
Self-hosted web scraper, crawler and document parser for AI agents: URLs, PDFs with OCR and Office files to clean Markdown/JSON. Firecrawl- and Jina-compatible API, MCP server, agent endpoint, Python/Node SDKs. Single Rust binary.
- ai-agents
- browser-automation
- data-extraction
- firecrawl