A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
- deepseek
- inference
- inference-engine
- llm-inference
111,821 visible repositories in the index
1,177–1,200 of 2,463 · Most stars
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)
An open-source AI platform for knowledge work. Connect your apps, find answers, and get work done.
A series of code large language models developed by PKU-KCL
Cache-Augmented Generation: A Simple, Efficient Alternative to RAG
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
The repository host gives no description.
Tensorflow implementation of contextualized word representations from bi-directional language models
A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
A fork to add multimodal model training to open-r1
ExtractThinker is a Document Intelligence library for LLMs, offering ORM-style interaction for flexible and powerful document workflows.
一行Docker命令部署的 OpenAI/GPT API代理,支持SSE流式返回、腾讯云函数 。Simple proxy for OpenAi api via a one-line docker command
Millisecond decisions, any domain: a 0.8B open "System 1" model that picks between your options with calibrated probabilities. One base, swappable LoRA adapters, on your own hardware.
An AI companion for reading papers.
Open Academic Research on Improving LLaMA to SOTA LLM
The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
Nativ is an open-source local AI app for Mac. It runs open AI models on Apple Silicon, and nothing leaves your Mac.
WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)
AgentEvolver: Towards Efficient Self-Evolving Agent System
An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
Stanford NLP Python library for Representation Finetuning (ReFT)
Chat language model that can use tools and interpret the results
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model