Main reference implementation for NLWeb, implemented in Python.
Repository Index
107,098 visible repositories in the index
49–72 of 238 · Most stars
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
- amd
- llama
- llm
- llm-inference
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
- llama
- llamacpp
- llm-inference
- ollama-api
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
- llava
- llm
- mlx
- vision-transformer
Robust recipes to align language models with human and AI preferences
- llm
- rlhf
- transformers
A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.
State of the art inference performance with modern accelerators on Kubernetes
A blazing fast inference solution for text embeddings models
Translatedcube studio is an open-source, cloud-native, one-stop machine learning / deep learning / large model AI platform. It covers the full mlops algorithm pipeline, a computing power rental platform, online notebook development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference services with VGPU virtualization, edge computing, a labeling platform with automated labeling, SFT fine-tuning / reward model / reinforcement learning training for large models such as deepseek, multi-machine large model inference with vllm/ollama/mindie, a private knowledge base, and an AI model marketplace. It supports domestic Chinese CPUs/GPUs/NPUs and the Ascend ecosystem, supports RDMA, and supports distributed frameworks such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano.
A fast inference library for running LLMs locally on modern consumer-class GPUs
TranslatedEnterprise-grade Agentic RAG agent - full-pipeline coverage of document parsing, multi-channel retrieval, intent recognition, question rewriting, conversation memory, MCP tool calling, and deep thinking. Built for real business scenarios, a complete engineering implementation from 0 to 1.
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Optimizing inference proxy for LLMs
- agent
- agentic-ai
- agentic-workflow
- agents
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
- ai
- amd-gpu
- apple-silicon
- asr
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
- apple-silicon
- local-llm
- macos
- mlx
The Apache Tika toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF).
- java
- tika
- metadata
- extraction
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
CSGHub is a brand-new open-source platform for managing LLMs, developed by the OpenCSG team. It offers both open-source and on-premise/SaaS solutions, with features comparable to Hugging Face. Gain full control over the lifecycle of LLMs, datasets, and agents, with Python SDK compatibility with Hugging Face. Join us! ⭐️
- ai
- huggingface
- llm
- management-system
TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients. Published in Nature.
- large-language-models
- prompt-optimization
- compound-systems
- textual-gradients
1 API key. 220,000+ AI models. Bytez is the largest serverless Model Inference API on the internet. Let's build ✨
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
A library for transfer learning by reusing parts of TensorFlow models.
- tensorflow
- machine-learning
- transfer-learning
- embeddings
Open-source inference server and production cluster for all the models your agent needs.
High-performance single-GPU inference for selected model checkpoints and GPUs.