Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
Repository Index
47,133 repositories indexed
1–24 of 30 · Most stars
👾 Fast and simple video download library and CLI tool written in Go
Elegant Scraper and Crawler Framework for Golang
Weighs the soul of incoming HTTP requests to stop AI crawlers
- defense
- security
A next-generation crawling and spidering framework.
- crawler
- web-spider
- gocrawler
- spider-framework
TranslatedDistributed web crawler admin platform for spiders management regardless of languages and frameworks.
- webcrawler
- scrapy
- crawlab
- spiders-management
A Chrome DevTools Protocol driver for web automation and scraping.
- cdp
- chrome-headless
- chrome-devtools
- chrome-devtools-protocol
Photo gallery for self-hosted personal servers
A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
- bittorrent
- dht
- prowlarr
- radarr
IoT Platform Framework
- golang
- iot-platform
- edge-computing
- distributed-systems
Extract structured data from web sites. Web sites scraping.
- golang
- golang-library
- extract-data
- chrome-fetcher
The free and privacy respecting pictionary game - Play at https://scribblers.bios-marcel.link
- montagsmaler
- drawing
- guessing-game
- multiplayer
teler-waf is a Go HTTP middleware that protects local web services from OWASP Top 10 threats, known vulnerabilities, malicious actors, botnets, unwanted crawlers, and brute force attacks.
- ids
- teler
- waf
- teler-ids
The unix-way web crawler
A small, fast code editor written in Go, with a native GPU-drawn UI (no webview, no Electron)
- code-editor
- go
- golang
- gpu
Library to use HTML5 Canvas from Go-WASM, with all drawing within go code
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- golang
- web-crawler
- crawling
- scraping
:wink: :cyclone: :strawberry: TextRank implementation in Golang with extendable features (summarization, phrase extraction) and multithreading (goroutine).
- go
- golang
- textrank
- summarization
Go package for sharding databases ( Supports every ORM or raw SQL )
- go
- golang
- golang-library
- database-sharding
A cross-platform native UI engine with a Go backend: write apps in Go with signals, memos and flexbox layout, and each platform draws with its own native views — iOS, macOS, Windows, Android, Linux and the web
Extract web archive data using Wayback Machine and Common Crawl
- golang
- crawler
- commoncrawl
- wayback-machine
Enterprise-grade 2D graphics library for Go. Pure Go, zero CGO. Part of GoGPU ecosystem.
- 2d-graphics
- creative-coding
- drawing
- go
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
- go
- golang
- goquery
- colly
A Grid based 2D Graphics library