Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
Repository Index
108,094 visible repositories in the index
121–144 of 244 · Most stars
[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation
- diffusion-models
- generative-model
- rlhf
- human-preferences
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Tika-Python is a Python binding to the Apache Tika™ REST services allowing Tika to be called natively in the Python community.
- tika-server
- python
- tika-python
- tika-server-jar
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Measuring Massive Multitask Language Understanding | ICLR 2021
- muti-task
- transfer-learning
- gpt-3
- few-shot-learning
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
- deepseek
- inference
- inference-engine
- llm-inference
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
ExtractThinker is a Document Intelligence library for LLMs, offering ORM-style interaction for flexible and powerful document workflows.
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
- deep-learning
- language-model
- llm
- machine-learning
Easily deployable and scalable backend server that efficiently converts various document formats (pdf, docx, pptx, html, images, etc) into Markdown. With support for both CPU and GPU processing, it is Ideal for large-scale workflows, it offers text/table extraction, OCR, and batch processing with sync/async endpoints.
- api
- fastapi
- markdown-parser
- pdf-chatbot
V100 / SM70-focused vLLM engineering fork for modern LLM inference.
Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.
- decision-models
- jev
- laya
- local-inference
💃 Dance with Intelligence in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.
Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
- amd
- gfx1151
- gpu-inference
- hip
A Docker-powered service for PDF document layout analysis. This service provides a powerful and flexible PDF analysis service. The service allows for the segmentation and classification of different parts of PDF pages, identifying the elements such as texts, titles, pictures, tables and so on.
KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
把英文论文读成舒服的中文:本地 PDF 论文翻译、原文对照、边读边问 AI、文献管理。Read English papers in comfortable Chinese.
- academic
- arxiv
- chinese
- claude-code
Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀
- machine-learning
- indexing
- rag
- rust
OpenTelemetry Instrumentation for AI Observability
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
- ai
- ai-art
- art
- asset-generator