MiniCPM-OCR

MiniCPM-V Multimodal Series

MiniCPM-OCR 1.0

Connect to a running recognition service before using these examples. The SDK and the inference backend use separate Python environments.

Document parsing

In the SDK environment, install the package and GPU layout detector:

Bash

pip install uv
uv venv .minicpm-ocr --python 3.12 --seed
source .minicpm-ocr/bin/activate
python -m pip install "minicpm-ocr[all]==1.0.2"
python -m pip install "paddlepaddle-gpu==3.3.0" \
  -i https://www.paddlepaddle.org.cn/packages/stable/cu129/

minicpmvocr parse \
  --device cuda:0 \
  --server-url http://127.0.0.1:18499 \
  --model minicpm-ocr \
  --input doc.pdf \
  --output doc.md

Run parsing from the cloned MiniCPM-OCR repository root. The SDK checks models/PP-DocLayoutV3_infer/ and downloads missing layout weights automatically. Replace the input with your PDF, PNG or JPG file; the output is Markdown with HTML tables and LaTeX formulas.

Region recognition

For an already cropped region, call the OpenAI-compatible service directly. Install openai and save the image as sample.png.

Bash

pip install openai

Python

import base64
from pathlib import Path
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:18499/v1", api_key="EMPTY")
image = base64.b64encode(Path("sample.png").read_bytes()).decode("utf-8")
response = client.chat.completions.create(
    model="minicpm-ocr",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image}"}},
        {"type": "text", "text": "Text Recognition:"},
    ]}],
    temperature=0.01,
    top_p=0.001,
    max_tokens=2048,
    stream=False,
)
print(response.choices[0].message.content)
print("finish_reason:", response.choices[0].finish_reason)

EMPTY is for the local service above without authentication. For an authenticated service, use its supplied URL, key and model ID. If finish_reason is length, check for truncation and increase the output budget within the service limit.

Region	Prompt
Text	Text Recognition:
Table	Table Recognition:
Formula	Formula Recognition:
Seal	Seal Recognition:
Chart	Chart to Table:

Key information extraction

Plain Text

Extract the "store_name", "amount". Format the output as a JSON object with keys: "store_name", "amount". If an item is not in the image, use "###".

Replace the recognition prompt with this instruction and keep both field lists consistent. Parse the generated JSON and validate values before downstream use.

MiniCPM-OCR Jev

After starting the service, send a cropped region and its OCR text. This example uses the renderer-enabled Docker service.

Score one region

Python

import base64
import json
import urllib.request
from pathlib import Path

image = base64.b64encode(Path("region.png").read_bytes()).decode("utf-8")
payload = {
    "input_form": "ternary",
    "items": [{
        "case_id": "blk-001",
        "image": f"data:image/png;base64,{image}",
        "ocr_text": "The OCR text to be scored",
        "category": "text",
    }],
}
request = urllib.request.Request(
    "http://127.0.0.1:8030/v1/rm/score",
    data=json.dumps(payload).encode("utf-8"),
    headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(request) as response:
    result = json.load(response)["results"][0]
if result["verdict"] == "scored":
    print(result["prob_yes"])
else:
    print(result["verdict"], result.get("detail", ""))

Set category to text, formula or table to match the region. A wrong category can distort scores without producing an error. Use the corresponding MIME prefix for JPEG images.

Read scores safely

Check verdict before reading prob_yes. Failed scoring may return fixed values 0.0 or -1.0; these are not valid low-confidence scores. Choose acceptance thresholds on your business data. Keep crop boundaries consistent when comparing candidates.

Related Resources

Topic	Link
Model Overview	MiniCPM-OCR
Deployment	MiniCPM-OCR
Official Documentation	SDK
API Access	OCR 1.0 / Jev