Deploying MiniCPM5-1B with OpenVINO™: an on-device 1B reasoning model and Hybrid Reasoning in practice
1. Introduction: bringing a reasoning model to the AI PC
OpenBMB has released MiniCPM5-1B (https://huggingface.co/openbmb/MiniCPM5-1B), the first model in the MiniCPM5 series. It has a dense 1B backbone and focuses on on-device deployment, agent tool calling, code generation, and difficult reasoning. Its standout feature is Hybrid Reasoning: the same weights can switch between "quick answer" and "think first, then answer" via the enable_thinking field in the chat template, avoiding the need to maintain multiple checkpoints for different scenarios.
This article provides a reference deployment path: install the environment → export INT4 IR with one optimum-cli command → run inference with openvino-genai LLMPipeline → try both Hybrid Reasoning modes. All commands and code in this article have been verified on Windows + Python 3.12 on Intel platforms, with local verification artifacts included. For the full example, see https://github.com/openvinotoolkit/openvino_notebooks/pull/3468.
2. MiniCPM5-1B model highlights: 2.1 Architecture specs: dense 1B / 24 layers / 131K context
Type: Causal Language Model, standard LlamaForCausalLM structure (dense, not MoE);
Parameters: 1,080,632,832 (about 1B), with 679M non-embedding parameters;
Layers: 24, attention GQA 16 Q heads / 2 KV heads;
Context length: 131,072 tokens, trained in BF16;
License: Apache-2.0, commercially usable.
2.2 Open-source SOTA at the same size
The model card reports that MiniCPM5-1B reaches 1B-class open-source SOTA in comparisons with LFM2.5-1.2B-Thinking, Qwen3-0.6B/think, and Qwen3.5-0.8B/think, with the largest leads in tool calling, code generation, and difficult reasoning.
Effects of RL + OPD (On-Policy Distillation) post-training:
Average score on math / code / instruction following +16 points
Proportion of "overly long outputs" hitting the max-tokens limit −29 percentage points
2.3 Hybrid Reasoning: one set of weights, two thinking modes
MiniCPM5-1B's chat template includes a built-in think block, controlled by enable_thinking passed to apply_chat_template: when enabled, the model first reasons explicitly in the think segment before answering; when disabled, it skips reasoning and answers directly. Official recommended sampling parameters: Think mode temperature=0.9, top_p=0.95; No-Think mode temperature=0.7, top_p=0.95.
3. Why OpenVINO™ + Optimum Intel + GenAI
Optimum Intel: a single optimum-cli export openvino --task text-generation-with-past converts an HF model to OpenVINO IR, with built-in NNCF INT4 / INT8 / FP16 weight compression.
The LLMPipeline in openvino-genai provides KV-Cache management, a streaming streamer, and ChatHistory out of the box. It mirrors the chat_sample.py example, so you can get a production-grade chat experience in a few lines of code.
All backends: the same IR model runs unchanged on Intel CPUs, Core Ultra iGPUs, and Arc discrete GPUs — only the device string changes.
4. End-to-end deployment of MiniCPM5-1B: 4.1 Environment setup
Python 3.10 or later is recommended. Create and activate a dedicated venv, then install the tutorial dependencies:
python -m venv minicpm5-venv
# Windows: minicpm5-venv\Scripts\activate
# Linux / macOS: source minicpm5-venv/bin/activate
pip install -U \
"openvino-genai" \
"git+https://github.com/huggingface/optimum-intel.git" \
"nncf>=3.0" \
"transformers>=5.6" \
"torch" \
"accelerate" \
"huggingface_hub"
MiniCPM5 is a newly released model, and some features require the optimum-intel main branch for full support, so use the git main branch first; switch back once a new PyPI release is available.
If access to Hugging Face is slow in mainland China, you can temporarily inject hf-mirror before the export command:
# Linux / macOS / Git Bash
export HF_ENDPOINT=https://hf-mirror.com
# Windows PowerShell
$env:HF_ENDPOINT = "https://hf-mirror.com"
4.2 One-command INT4 quantized IR export
INT4 weight compression is recommended — after quantization, MiniCPM5-1B is about 800 MB, so a 1B dense model can run smoothly on an AI PC:
optimum-cli export openvino \
--model openbmb/MiniCPM5-1B \
--task text-generation-with-past \
--weight-format int4 \
--group-size 128 \
--ratio 0.8 \
MiniCPM5-1B-ov/INT4
--task text-generation-with-past tells optimum-intel this is a causal language model with KV-Cache; --ratio 0.8 means 80% of weights use INT4 and the remaining 20% keep higher precision, a default trade-off the community has found stable for LLMs; --group-size 128 is the NNCF group size for grouped quantization.
After export, the directory structure is:
MiniCPM5-1B-ov/INT4/
├── chat_template.jinja
├── config.json
├── generation_config.json
├── openvino_config.json
├── openvino_detokenizer.bin / .xml
├── openvino_model.bin (~822 MB) / .xml
├── openvino_tokenizer.bin / .xml
├── tokenizer.json
└── tokenizer_config.json
For higher quality or a smaller footprint, respectively:
# INT8
optimum-cli export openvino --model openbmb/MiniCPM5-1B \
--task text-generation-with-past --weight-format int8 \
MiniCPM5-1B-ov/INT8
# FP16
optimum-cli export openvino --model openbmb/MiniCPM5-1B \
--task text-generation-with-past --weight-format fp16 \
MiniCPM5-1B-ov/FP16
4.3 Reproducing chat_sample.py: a minimal runnable example
The code below is a minimized version of OpenVINO GenAI's official chat_sample.py (https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/python/text_generation/chat_sample.py), paired with the same prompt used in the transformers section of the Hugging Face model card (https://huggingface.co/openbmb/MiniCPM5-1B#transformers): Who are you? Please briefly introduce yourself.
import openvino_genai
MODEL_DIR = "MiniCPM5-1B-ov/INT4"
DEVICE = "CPU" # "GPU" / "AUTO" also work
pipe = openvino_genai.LLMPipeline(MODEL_DIR, DEVICE)
config = openvino_genai.GenerationConfig()
config.max_new_tokens = 128
def streamer(subword: str) -> openvino_genai.StreamingStatus:
print(subword, end="", flush=True)
return openvino_genai.StreamingStatus.RUNNING
# Same as chat_sample.py: use ChatHistory to accumulate multi-turn context
history = openvino_genai.ChatHistory()
history.append({
"role": "user",
"content": "Who are you? Please briefly introduce yourself.",
})
result = pipe.generate(history, config, streamer)
print("\n----------")
After running, the model should stream a self-introduction, and the full answer is available in result.texts[0]. Local test output excerpt (INT4 + Intel CPU):
<think>Hmm, the user is asking about my identity and wants a brief introduction....think>
I'm a model from the MiniCPM series, developed by ModelBest Inc. and the Open...
By default, the chat template enables thinking, so the output shows a thinking block — in the next section we use enable_thinking to control this behavior explicitly.
4.4 Multi-turn chat loop (same structure as the official example)
Wrapping 4.3 in an interactive loop gives the standard form of chat_sample.py:
import openvino_genai
pipe = openvino_genai.LLMPipeline("MiniCPM5-1B-ov/INT4", "CPU")
config = openvino_genai.GenerationConfig()
config.max_new_tokens = 256
def streamer(subword):
print(subword, end="", flush=True)
return openvino_genai.StreamingStatus.RUNNING
history = openvino_genai.ChatHistory()
while True:
try:
prompt = input("\nquestion:\n")
except EOFError:
break
history.append({"role": "user", "content": prompt})
result = pipe.generate(history, config, streamer)
history.append({"role": "assistant", "content": result.texts[0]})
print("\n----------")
5. Hybrid Reasoning: think vs. no-think in practice
openvino-genai provides the extra_context parameter of Tokenizer.apply_chat_template, which is exactly what we need to pass enable_thinking through to the model's built-in chat template and switch precisely between the two modes:
import openvino_genai
pipe = openvino_genai.LLMPipeline("MiniCPM5-1B-ov/INT4", "CPU")
tokenizer = pipe.get_tokenizer()
messages = [
{"role": "user",
"content": "Who are you? Please briefly introduce yourself."},
]
def streamer(subword):
print(subword, end="", flush=True)
return openvino_genai.StreamingStatus.RUNNING
for mode, enable_thinking, T in [("NO-THINK", False, 0.7),
("THINK", True, 0.9)]:
print(f"=== {mode} ===")
prompt = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
extra_context={"enable_thinking": enable_thinking},
)
config = openvino_genai.GenerationConfig()
config.max_new_tokens = 256
config.do_sample = True
config.temperature = T
config.top_p = 0.95
pipe.generate(prompt, config, streamer)
print()
In local tests, the tail of the prompt generated by apply_chat_template differs noticeably between the two modes, which determines model behavior:
# enable_thinking=False -> the assistant segment is prefilled with an empty think block
'... <|im_start|>assistant\n\n\n\n\n'
# enable_thinking=True -> leave the think block empty for the model to expand freely
'... <|im_start|>assistant\n\n'
Measured output excerpts (INT4 + CPU + the sampling parameters above):
=== NO-THINK ===
<think>
think>
I am a MiniCPM series model, developed by ModelBest and the OpenBMB community. For more information about the project, visit https://github.com/OpenBMB/.
=== THINK ===
<think>...think>
I am a MiniCPM series model, developed by ModelBest and the OpenBMB open-source community. ...
When to use No-Think: casual Q&A, customer service FAQ, pure execution tasks — latency first, shorter outputs.
When to use Think: math, code, complex planning — quality first; raise max_new_tokens above 1024 and enable streaming to avoid the feeling of waiting.
6. Performance and precision recommendations
INT4 is about 1/4 the size of FP16; MiniCPM5-1B INT4 weights are ~820 MB, and a CPU alone can run inference smoothly;
If Think mode produces reasoning that is too short or empty on some prompts, raise temperature and increase max_new_tokens; for quantization-sensitive tasks, fall back to INT8 / FP16 and retest.
For long-context applications, remember to control KV-Cache memory via GenAI's scheduler_config in addition to GenerationConfig.
7. Summary and resources
MiniCPM5-1B brings "on-device 1B + Hybrid Reasoning" to open-source SOTA; the OpenVINO + Optimum Intel + GenAI combination lets you run production-grade multi-turn chat on Intel CPU / iGPU / discrete GPU with one optimum-cli command and a few dozen lines of Python, switching between the two thinking modes at will.
MiniCPM5-1B model card:
https://huggingface.co/openbmb/MiniCPM5-1B
OpenVINO notebook PR: https://github.com/openvinotoolkit/openvino_notebooks/pull/3468
OpenBMB/MiniCPM repository: https://github.com/OpenBMB/MiniCPM
openvino.genai chat_sample.py: https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/python/text_generation/chat_sample.py
Optimum Intel: https://github.com/huggingface/optimum-intel
OpenVINO: https://github.com/openvinotoolkit/openvino
OpenVINO GenAI: https://github.com/openvinotoolkit/openvino.genai
NNCF: https://github.com/openvinotoolkit/nncf
This workflow has been fully verified on Windows + Python 3.12 + Intel CPU: openvino-genai 2026.1, optimum-intel main branch, nncf 3.1, transformers 5.9; the exported INT4 IR is ~822 MB, and both single-prompt + ChatHistory inference and the two Hybrid Reasoning modes ran successfully.
OpenVINO™
*OpenVINO™ and the OpenVINO™ logo are trademarks of Intel Corporation or its subsidiaries.
OpenVINO assistant on WeChat
WeChat ID: OpenVINO-China
For inquiries or related discussions, add the OpenVINO assistant on WeChat to join the dedicated community and interact with technical experts in real time.