MiniCPM5-2B
MiniCPM Text Series
Deployment Cookbook and Agent Skills
MiniCPM5-2B uses the standard LlamaForCausalLM architecture, so mainstream inference engines can load it directly, with no custom kernels and no forked model code. To help developers quickly reproduce the deployment process, we provide a single-page cookbook, each section paired one-to-one with an Agent Skill.
Cookbook: suited to running steps manually. Agent Skills: suited to letting an agent in a coding agent such as Cursor / Claude Code choose the execution route based on the target backend, hardware, and data paths.
Top-level Skill
MiniCPM5 Deploy Skill (automatically routes to transformers · vllm · sglang · llama-cpp · ollama · lmstudio · mlx · arclight · vllm-ascend)
After downloading this Skill and placing it in your current project directory, you can type the following in a coding agent such as Cursor / Claude Code:
@minicpm5-deploy 用 vLLM 在 8000 端口启动 openbmb/MiniCPM5-2B
The agent reads the top-level Skill, selects the matching sub-Skill and cookbook based on the target backend, hardware, and data paths, then runs the commands and reports the results.
Quantized models are not listed as separate backends; they are covered under the inference backends that can load the corresponding formats.
Inference Deployment
| Backend | Models / Use case | Cookbook | Agent Skill |
|---|---|---|---|
| Transformers | BF16 / FP16, local Python inference, GPU + CPU | Get it | Get it |
| vLLM | BF16 / FP16 OpenAI server | Get it | Get it |
| SGLang | BF16 / FP16 OpenAI server, recommended for tool calling and DSpark | Get it | Get it |
| llama.cpp | GGUF, local CPU / GPU inference | Get it | Get it |
| Ollama | GGUF, local on-device use | Get it | Get it |
| LM Studio | GGUF, Mac desktop app and OpenAI server | Get it | Get it |
| MLX | MLX / 4bit, local inference on Apple Silicon | Get it | Get it |
| ArcLight | GGUF, local on-device / CPU / desktop / server | Get it | Get it |
| vLLM Ascend | BF16 / FP16 OpenAI server, Ascend devices | Get it | Get it |
Other Supported Frameworks
In addition to the deployment frameworks listed above, MiniCPM5-2B also supports multi-chip deployment via FlagOS.
FlagOS provides a unified multi-chip AI system software stack and has released multi-chip versions of MiniCPM5-2B on FlagRelease, covering platforms including Nvidia, Hygon, MetaX, Iluvatar CoreX, Enflame, Moore Threads, Kunlunxin, Ascend, and ARM-v9.
View the MiniCPM5-2B FlagOS multi-chip versions and full usage instructions