LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Repository Index
61,275 visible repositories in the index
1–8 of 8 · Most stars
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
System prompt builder. Assembles agent identity from workspace config and spark.toml. Part of Unicity AOS.
Translatedcube studio is an open-source, cloud-native, one-stop machine learning / deep learning / large model AI platform. It covers the full mlops algorithm pipeline, a computing power rental platform, online notebook development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference services with VGPU virtualization, edge computing, a labeling platform with automated labeling, SFT fine-tuning / reward model / reinforcement learning training for large models such as deepseek, multi-machine large model inference with vllm/ollama/mindie, a private knowledge base, and an AI model marketplace. It supports domestic Chinese CPUs/GPUs/NPUs and the Ascend ecosystem, supports RDMA, and supports distributed frameworks such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano.
Prompt Orchestration Markup Language
- llm
- markup-language
- prompt
- vscode-extension
TranslatedA visual management tool for the OpenAI Codex desktop app/CLI, offering Provider/API switching, session sync, prompt injection, Skills/MCP management, and TOML configuration visualization. It is a cross-platform tool.
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
- apple-silicon
- local-llm
- macos
- mlx
OML 1.0 via Fingerprinting: Open, Monetizable, and Loyal AI
- oml
- fine-tuning
- fingerprint
- loyalty