Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.
Repository Index
87,573 visible repositories in the index
169–192 of 904 · Most stars
A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Providing a free OpenAI GPT-4 API ! This is a replication project for the typescript version of xtekky/gpt4free
Open Source LLM Gateway 🦚
- openclaw
- cost-tracking
- llm-observability
- token-tracking
AI Product Design Agent - Open Source
ClawX is a desktop app that provides a graphical interface for OpenClaw AI agents. It turns CLI-based AI orchestration into a desktop experience without using the terminal. China website is https://clawx.com.cn.
A pixel office for your OpenClaw: turn invisible work states into a cozy little space with characters, daily notes, and guest agents. Code under MIT; art assets for non-commercial learning only.
- agent-collaboration
- ai-assistant
- dashboard
- flask
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
- llmops
- observability
- open-telemetry
- metrics
A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
- agents
- guardrails
- python
- safety
💼 An enterprise-grade Next.js boilerplate for high-performance, maintainable apps. Packed with features like Tailwind CSS, TypeScript, ESLint, Prettier, testing tools, and more to accelerate your development.
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
- gemma
- gpgpu
- llm
- local-ai
PDF GPT allows you to chat with the contents of your PDF file by using GPT capabilities. The most effective open source solution to turn your pdf files in a chatbot!
- pdfgpt
- chatpdf
- chatwithpdf
Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
- gateway
- generative-ai
- llm-inference
- llms
Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it.
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
- deep-learning
- pytorch
- machine-translation
- neural-machine-translation
An APP that integrates mainstream large language models and image generation models, built with Flutter, with fully open-source code.
Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.
Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.
- localllm
Native web workspace for Hermes Agent — chat, terminal, memory, skills, inspector.
- agent-ui
- ai-workspace
- hackathon
- hermes-agent
Point at any URL/YouTube/Podcast or file. Get the gist. CLI and Chrome Extension.
One UI is all done with chatgpt web, midjourney, gpts,suno,luma,runway,viggle,flux,ideogram,realtime,pika,udio; Simultaneous support Web / PWA / Linux / Win / MacOS platform
- chatgpt-ui
- midjourney-ui
- gpts
- suno