MiniCPM-V 4.7

MiniCPM-V Multimodal Series

Bring the model into your own service: vLLM and SGLang target high-throughput GPU serving, llama.cpp and Ollama target local and on-device runs. Pick one based on your hardware and scenario; this page covers installation, startup and invocation for each.

MiniCPM-V 4.7 ships as two sizes; every framework selects one via the model path:

Version	Params	HuggingFace	ModelScope
MiniCPM-V 4.7 1B (dense)	1B	MiniCPM-V-4.7-1B	MiniCPM-V-4.7-1B
MiniCPM-V 4.7 35B-A3B (MoE)	35B-A3B	MiniCPM-V-4.7-35B-A3B	MiniCPM-V-4.7-35B-A3B

Framework	Status	Best for
vLLM	Supported on upstream (PR #58674)	High-throughput GPU serving
SGLang	Uses fork `tc-mb/sglang`, upstream PR in progress	High-throughput GPU serving
llama.cpp	Supported on upstream (PR #29416)	Local CPU / Metal
Ollama	Model code ready, runtime awaiting built-in llama.cpp sync	Quick local trial

Shared prerequisites

v4.7 is registered as MiniCPMV4_7ForConditionalGeneration and requires transformers >= 5.18.0; all frameworks work without trust_remote_code. v4.7 uses the Qwen3.5 vocabulary, with stop tokens [248044, 248046] (v4.5 used [1, 151645]).

vLLM

MiniCPM-V 4.7 is supported on the official vLLM main branch since PR #58674. No fork is required — install directly from upstream, making sure your checkout is at or after that commit.

Environment Setup

Bash

# Create a clean conda environment
conda create -n vllm-v47 python=3.10 -y
conda activate vllm-v47

# Install from upstream main (requires CUDA toolkit; HEAD must include #58674)
git clone https://github.com/vllm-project/vllm.git
cd vllm
MAX_JOBS=6 VLLM_USE_PRECOMPILED=1 pip install --editable . -v

Once a vLLM release containing #58674 is published, you can simply pip install -U vllm and skip the source build.

For video inference, install the video module:

Bash

pip install vllm[video]

Verify the installation:

Bash

python -c "import vllm, transformers; print('vllm', vllm.__version__, '| transformers', transformers.__version__)"

Launch API Service

Bash

vllm serve <model_path> \
  --dtype auto \
  --max-model-len 8192 \
  --api-key token-abc123 \
  --gpu_memory_utilization 0.9 \
  --max-num-batched-tokens 8192

<model_path>: Local path, or openbmb/MiniCPM-V-4.7-1B / openbmb/MiniCPM-V-4.7-35B-A3B.
--api-key: API access key.
--max-model-len: Maximum context length (v4.7 supports up to 256K, start small to fit GPU memory).
--gpu_memory_utilization: GPU memory utilization rate.

Image Inference

Python

from openai import OpenAI
import base64

openai_api_key = "token-abc123"   # must match the value passed to vllm serve
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(api_key=openai_api_key, base_url=openai_api_base)

with open('./assets/airplane.jpeg', 'rb') as file:
    image = "data:image/jpeg;base64," + base64.b64encode(file.read()).decode('utf-8')

chat_response = client.chat.completions.create(
    model="<model_path>",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": image}},
        ],
    }],
    extra_body={
        # v4.7 uses Qwen3.5 vocab, stop tokens same as v4.6
        "stop_token_ids": [248044, 248046],
        # Disable <think> blocks for direct answers
        "chat_template_kwargs": {"enable_thinking": False},
    }
)
print(chat_response.choices[0].message.content)

Video Inference

Python

from openai import OpenAI
import base64

openai_api_key = "token-abc123"
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(api_key=openai_api_key, base_url=openai_api_base)

with open('./videos/video.mp4', 'rb') as video_file:
    video_base64 = base64.b64encode(video_file.read()).decode('utf-8')

chat_response = client.chat.completions.create(
    model="<model_path>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this video"},
                {"type": "video_url", "video_url": {"url": f"data:video/mp4;base64,{video_base64}"}},
            ],
        },
    ],
    extra_body={"stop_token_ids": [248044, 248046]},
)
print(chat_response.choices[0].message.content)

Adjust the frame slice budget per video:

Python

extra_body = {
    "stop_token_ids": [248044, 248046],
    "mm_processor_kwargs": {"max_slice_nums": 2},
}

Multiple Images / Videos

Raise the --limit-mm-per-prompt budget at startup:

Bash

# Up to 5 images + 2 videos per request
vllm serve <model_path> --dtype auto --max-model-len 16384 --api-key token-abc123 \
  --gpu_memory_utilization 0.9 \
  --limit-mm-per-prompt '{"image": 5, "video": 2}'

Offline Inference

Python

from transformers import AutoProcessor
from PIL import Image
from vllm import LLM, SamplingParams

MODEL_NAME = "<model_path>"   # or "openbmb/MiniCPM-V-4.7-1B"
image = Image.open("./assets/airplane.jpeg").convert("RGB")

processor = AutoProcessor.from_pretrained(MODEL_NAME)
llm = LLM(
    model=MODEL_NAME,
    max_model_len=8192,
    disable_mm_preprocessor_cache=True,
    limit_mm_per_prompt={"image": 5},
)

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "Describe the content of this image"},
    ],
}]
prompt = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

inputs = {"prompt": prompt, "multi_modal_data": {"image": image}}
sampling_params = SamplingParams(
    stop_token_ids=[248044, 248046],
    temperature=0.7,
    top_p=0.8,
    max_tokens=1024,
)

outputs = llm.generate(inputs, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)

Notes

GPU Memory: 1B fits on a single consumer card; 35B-A3B (bf16 ~70 GB) needs multi-GPU or 4-bit weights (-AWQ / -GPTQ / -FP8 / -INT4). Adjust --gpu_memory_utilization / --max-model-len / --max-num-batched-tokens based on hardware.
Multi-modal limits: For multi-image / multi-video conversations, use --limit-mm-per-prompt.

SGLang

MiniCPM-V 4.7 has not yet been merged into upstream SGLang. Please use the fork tc-mb/sglang on the Support-MiniCPM-V4.7 branch. An upstream PR is in progress; once merged, you can install SGLang main directly.

Installation

Bash

git clone -b Support-MiniCPM-V4.7 https://github.com/tc-mb/sglang.git
cd sglang

pip install --upgrade pip
pip install -e "python[all]"

Verify the environment:

Bash

python -c "import torch, transformers; print('torch', torch.__version__, '| cuda', torch.version.cuda, '| transformers', transformers.__version__)"

Install FlashInfer (Optional, Recommended)

FlashInfer wheels are tied to specific (torch, cuda) combinations. Pick the index that matches your torch + CUDA from the verification above — don't blindly copy cu121/torch2.4 URLs.

General indices are at flashinfer.ai/whl, for example:

Your torch / CUDA	Index URL
torch 2.7 + CUDA 12.8	`https://flashinfer.ai/whl/cu128/torch2.7/`

Bash

pip install flashinfer-python -i <corresponding index URL from table>

See the SGLang official installation docs for Docker images and CPU-only fallbacks.

Launch Inference Service

Bash

# 1B
python -m sglang.launch_server --model-path openbmb/MiniCPM-V-4.7-1B --port 30000 --dtype bfloat16

# 35B-A3B (requires large-memory node; use --tp-size for multi-GPU sharding)
python -m sglang.launch_server --model-path openbmb/MiniCPM-V-4.7-35B-A3B --port 30000 --dtype bfloat16 --tp-size 2

Or specify a local path:

Bash

python -m sglang.launch_server --model-path /your/local/MiniCPM-V-4.7-1B --port 30000 --dtype bfloat16

Call the Service

curl:

Bash

curl -s http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniCPM-V-4.7-1B",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "What is in this image?"},
          {
            "type": "image_url",
            "image_url": {
              "url": "https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/inference/assets/airplane.jpeg?raw=true"
            }
          }
        ]
      }
    ],
    "max_tokens": 300,
    "top_k": -1
  }'

Python (OpenAI client):

Python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="None")

response = client.chat.completions.create(
    model="MiniCPM-V-4.7-1B",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/inference/assets/airplane.jpeg?raw=true",
                    },
                },
            ],
        }
    ],
    max_tokens=300,
    extra_body={"top_k": -1, "stop_token_ids": [248044, 248046]},
)

print(response.choices[0].message.content)

If image_url is not reachable, replace it with a local path / base64 data URL.
If the model continues to generate beyond the answer, pass stop_token_ids = [248044, 248046].
To disable <think> blocks, pass chat_template_kwargs={"enable_thinking": false} (or the server-side equivalent).

llama.cpp

MiniCPM-V 4.7 has been merged into the official llama.cpp (PR #29416). Please use a checkout that includes (or is after) that PR.

GGUF format changes

v4.7 uses canvas M-RoPE, introducing new GGUF metadata keys (rope.section_order) and mmproj tensors (v.tok_embd_sep). GGUF files converted with older convert_hf_to_gguf.py will load in recent builds but compute M-RoPE incorrectly (or fail due to missing mmproj tensors). Always use the official MiniCPM-V 4.7 GGUF, or reconvert with the current scripts.

Each size uses its own set of GGUF:

Size	LLM file	mmproj
1B	`MiniCPM-V-4_7-1B-Q4_K_M.gguf`	`mmproj-model-f16.gguf`
35B-A3B	`MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf`	`mmproj-model-f16.gguf`

Build llama.cpp

Bash

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp

Build with CMake (see the build docs for details):

Bash

# CPU / Metal
cmake -B build
cmake --build build --config Release

# CUDA
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

Get GGUF Files

Download the official GGUF (LM GGUF and vision projection mmproj):

HuggingFace: MiniCPM-V-4.7-1B-gguf · MiniCPM-V-4.7-35B-A3B-gguf
ModelScope: OpenBMB/MiniCPM-V-4.7-1B-gguf / OpenBMB/MiniCPM-V-4.7-35B-A3B-gguf

Each repo provides the full K-quant levels (F16, Q8_0, Q6_K, Q5_K_M, Q5_K_S, Q5_1, Q5_0, Q4_K_M, Q4_K_S, Q4_1, Q4_0) plus two mmproj files (mmproj-model-f16.gguf, mmproj-model-f16-4x.gguf). The 35B-A3B repo also includes mix_q3gud_im (14.4 GiB imatrix mixed quantization, recommended for memory-constrained devices).

You can also convert from PyTorch weights yourself; see the quantization section for steps.

Model Inference

Recent llama.cpp builds default --reasoning to auto, which enables thinking based on the chat template. The MiniCPM-V 4.7 template enables thinking by default, so for plain Q&A, explicitly pass --reasoning off.

Bash

cd build/bin/

# F16 weights (1B)
./llama-mtmd-cli \
    -m  /path/to/MiniCPM-V-4.7-1B/MiniCPM-V-4_7-1B-F16.gguf \
    --mmproj /path/to/MiniCPM-V-4.7-1B/mmproj-model-f16.gguf \
    -c 8192 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 \
    --reasoning off \
    --image xx.jpg -p "What is in the image?"

# Quantized INT4 weights (35B-A3B)
./llama-mtmd-cli \
    -m  /path/to/MiniCPM-V-4.7-35B-A3B/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
    --mmproj /path/to/MiniCPM-V-4.7-35B-A3B/mmproj-model-f16.gguf \
    -c 8192 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 \
    --reasoning off \
    --image xx.jpg -p "What is in the image?"

# Interactive mode
./llama-mtmd-cli \
    -m  /path/to/MiniCPM-V-4.7-1B/MiniCPM-V-4_7-1B-Q4_K_M.gguf \
    --mmproj /path/to/MiniCPM-V-4.7-1B/mmproj-model-f16.gguf \
    -c 8192 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 \
    --reasoning off \
    --image xx.jpg -i

To enable thinking (heavy OCR / reasoning tasks), use --reasoning on (--jinja is on by default):

Bash

./llama-mtmd-cli \
    -m  /path/to/MiniCPM-V-4.7-1B/MiniCPM-V-4_7-1B-Q4_K_M.gguf \
    --mmproj /path/to/MiniCPM-V-4.7-1B/mmproj-model-f16.gguf \
    -c 8192 --image xx.jpg --jinja --reasoning on -p "What is in the image?"

Parameter reference:

Parameter	Description
`-m, --model`	Path to the LM GGUF
`--mmproj`	Path to the vision projection GGUF
`--image`	Input image path
`-p, --prompt`	Prompt text
`-c, --ctx-size`	Maximum context length
`-rea, --reasoning [on\|off\|auto]`	Enable thinking. Default is `auto` (reads template). Use `off` for plain Q&A
`--jinja`	Use the model's Jinja chat template (enabled by default in recent builds)

Ollama

Ollama's model code recognizes MiniCPM-V 4.7 (convert_minicpmv.go), but its built-in llama.cpp must first sync to PR #29416 to load MiniCPM-V 4.7 GGUF (the new canvas M-RoPE format introduces rope.section_order and mmproj tensor v.tok_embd_sep). Until then, ollama create / ollama run with MiniCPM-V 4.7 GGUF will fail. Use recent llama.cpp directly for now, or wait for an Ollama release with the updated built-in engine.

Install Ollama

macOS: Ollama.dmg
Windows: OllamaSetup.exe
Linux: curl -fsSL https://ollama.com/install.sh | sh
Docker: ollama/ollama on Docker Hub

Run from Registry

Once an official release with the updated built-in engine ships, pull the official model from the Ollama registry:

Bash

ollama run openbmb/minicpm-v4.7

On the command line, separate the prompt from the image path with a space:

Plain Text

What is in the picture? xx.jpg

API call:

Python

import base64, requests

with open("xx.jpg", "rb") as image_file:
    encoded_string = base64.b64encode(image_file.read()).decode("utf-8")

data = {
    "model": "openbmb/minicpm-v4.7",
    "prompt": "What is in the picture?",
    "stream": False,
    "images": [encoded_string],   # Can include multiple base64 images
}
response = requests.post("http://localhost:11434/api/generate", json=data)

Load Local GGUF

To load local GGUF (e.g., custom quantization or a specific size), create a Modelfile.

Bash

vim minicpmv4.7.Modelfile

Modelfile content (1B example):

Plain Text

FROM ./MiniCPM-V-4.7-1B-gguf/MiniCPM-V-4_7-1B-Q4_K_M.gguf
FROM ./MiniCPM-V-4.7-1B-gguf/mmproj-model-f16.gguf

TEMPLATE """{{- if .Messages }}{{- range $i, $_ := .Messages }}{{- $last := eq (len (slice $.Messages $i)) 1 -}}<|im_start|>{{ .Role }}{{ .Content }}{{- if $last }}{{- if (ne .Role "assistant") }}<|im_end|><|im_start|>assistant{{ end }}{{- else }}<|im_end|>{{ end }}{{- end }}{{- else }}{{- if .System }}<|im_start|>system{{ .System }}<|im_end|>{{ end }}{{ if .Prompt }}<|im_start|>user{{ .Prompt }}<|im_end|>{{ end }}<|im_start|>assistant{{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}"""

SYSTEM """You are a helpful assistant."""

PARAMETER top_p 0.8
PARAMETER num_ctx 8192
PARAMETER stop ["<|im_start|>", "<|im_end|>"]
PARAMETER temperature 0.7

First FROM	Second FROM	num_ctx
LM GGUF path	Vision projection GGUF path	Maximum context length

For 35B-A3B, change the first FROM to MiniCPM-V-4.7-35B-A3B-gguf/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf.

Create and run:

Bash

ollama create minicpm-v4.7 -f minicpmv4.7.Modelfile
ollama run minicpm-v4.7

Notes

Template: MiniCPM-V 4.7's chat_template.jinja is byte-for-byte identical to v4.6, so the TEMPLATE above matches the one used by openbmb/minicpm-v4.6.
No RENDERER: Consistent with the official openbmb/minicpm-v4.6 (family qwen35, no renderer field), Ollama runs in native mode, using the GGUF embedded chat template + mmproj.
Thinking: Enabled by default in the template; to get direct answers, pass enable_thinking: false in the request (or use a runtime that supports chat_template_kwargs).

Developer Showcase

View all →

vLLM​

Environment Setup​

Launch API Service​

Image Inference​

Video Inference​

Multiple Images / Videos​

Offline Inference​

Notes​

SGLang​

Installation​

Install FlashInfer (Optional, Recommended)​

Launch Inference Service​

Call the Service​

llama.cpp​

Build llama.cpp​

Get GGUF Files​

Model Inference​

Ollama​

Install Ollama​

Run from Registry​

Load Local GGUF​

Notes​

Developer Showcase

vLLM

Environment Setup

Launch API Service

Image Inference

Video Inference

Multiple Images / Videos

Offline Inference

Notes

SGLang

Installation

Install FlashInfer (Optional, Recommended)

Launch Inference Service

Call the Service

llama.cpp

Build llama.cpp

Get GGUF Files

Model Inference

Ollama

Install Ollama

Run from Registry

Load Local GGUF

Notes