MiniCPM5-1B

MiniCPM Text Series

Deployment Cookbook and Agent Skills

MiniCPM5-1B uses the standard LlamaForCausalLM architecture, so mainstream inference engines can load it directly, with no custom kernels and no forked model code. To help developers quickly reproduce the deployment process, we provide a single-page cookbook, each section paired one-to-one with a Cursor Agent Skill.

Cookbook: suited to running steps manually. Agent Skills: suited to letting an agent in Cursor / Claude Code choose the execution route based on the target backend, hardware, and data paths.

Top-level Skill

MiniCPM5-1B Deploy Skill ( automatically routes to transformers · vllm · sglang · llama-cpp · ollama · lmstudio · mlx )

After downloading this Skill and placing it in your current project directory, you can type the following in a coding agent such as Cursor / Claude Code:

Plain Text

@minicpm5-deploy   用 vLLM 在 8000 端口起 openbmb/MiniCPM5-1B

The agent reads the top-level Skill, selects the matching sub-Skill and cookbook based on the target backend, hardware, and data paths, then runs the commands and reports the results.

The cookbooks and sub-Skills are listed below by inference backend.

Quantized models are not listed as separate backends; they are covered under the inference backends that can load the corresponding formats.

Inference Deployment

Backend	Models / Use case	Cookbook	Agent Skill
Transformers	BF16 / FP16, local Python inference, GPU + CPU	Get it	Get it
vLLM	BF16 / FP16 OpenAI server	Get it	Get it
SGLang	BF16 / FP16 OpenAI server, recommended for tool calling	Get it	Get it
llama.cpp	GGUF, local CPU/GPU inference	Get it	Get it
Ollama	GGUF, local on-device use	Get it	Get it
LM Studio	GGUF, Mac desktop app and OpenAI server	Get it	Get it
MLX	MLX / 4bit, local inference on Apple Silicon	Get it	Get it
ArcLight	GGUF, local on-device / CPU / desktop / server	Get it	Get it

---

Other Supported Frameworks

In addition to the deployment frameworks listed above, MiniCPM5-1B also supports multi-chip deployment via FlagOS.

Multi-AI-Chip Support

Thanks to the unified multi-chip AI system software stack of FlagOS, MiniCPM5-1B was adapted to 9 different AI chips in a very short time. Multi-chip versions of MiniCPM5-1B have now been released on FlagRelease, a platform built by the FlagOS team for automatic migration, adaptation, and release of large models across multi-architecture AI chips. If you're interested, jump here 👉 Learn more

Usage

Accelerating Performance on Nvidia with FlagOS

1. From FlagRelease (recommended)

FlagRelease is a platform built by the FlagOS team for automatic migration, adaptation, and release of large models across multi-architecture AI chips, and it has released multi-chip versions of MiniCPM-1B. FlagRelease ships with the relevant packages built in, so no installation is required.

1. From scratch

Requires a Python3.12, GLIBC_2.39, GLIBCXX_3.4.33, CXXABI_1.3.15 environment

Step 1: Install the FlagOS operator library

Official repository: https://github.com/flagos-ai/FlagGems

Bash

pip install flag-gems==4.2.1rc0
pip install triton==3.5.1

Step 2: Enable acceleration

Enable flagGems acceleration by adding the flagGems import to the vllm source code that runs inference:

Python

import flag_gems
flag_gems.enable(record=True, once=True, path="/root/gems.txt")

Bash

vllm serve ${model_path} \
--trust-remote-code \
--dtype bfloat16 \
--enforce-eager \
--port ${Port} \
--served-model-name ${model_name} \
--gpu-memory-utilization 0.85

Using the FlagOS Unified Multi-chip Backend Plugin

vllm-plugin-FL is a plugin built for the vLLM inference/serving framework. It is developed on FlagOS's unified multi-chip backend and aims to extend vLLM's functionality and performance across a variety of hardware environments.

Vendor	From scratch	From FlagRelease	From FlagRelease
Nvidia	vllm-plugin-FL/MiniCPM5-1B	MiniCPM5-1B-ModelScope	MiniCPM5-1B-nvidia-FlagOS