MiniCPM-OCR

MiniCPM-V Multimodal Series

Model Downloads

Version	Params	HuggingFace	ModelScope
MiniCPM-OCR 1.0	1.3B	MiniCPM-OCR-1.0	MiniCPM-OCR-1.0
MiniCPM-OCR 1.0-DSpark	0.3B	MiniCPM-OCR-1.0-DSpark	MiniCPM-OCR-1.0-DSpark
MiniCPM-OCR Jev	2.2B	MiniCPM-OCR-Jev	MiniCPM-OCR-Jev

MiniCPM-OCR 1.0

The document parser has two processes: the SDK handles layout and assembly, and the recognition service runs the model. This page starts the recognition backend; then follow Inference to parse a document.

Environment

The published vLLM recipe uses Linux x86_64, Python 3.12, an NVIDIA GPU and driver ≥580 (CUDA 13), verified on RTX 5090 32 GB. Reserve at least 40 GB disk for environments, source builds, weights and caches. This verified configuration is not a minimum VRAM estimate.

vLLM

Clone the repository and use its pinned installation script:

Bash

git clone https://github.com/OpenBMB/MiniCPM-OCR.git
cd MiniCPM-OCR
apt-get update
apt-get install -y software-properties-common
add-apt-repository -y ppa:ubuntu-toolchain-r/test
apt-get update
apt-get install -y gcc-13 g++-13 libdw-dev libelf-dev
bash scripts/install.sh vllm

Open a new terminal at the repository root:

Bash

source .venv-vllm/bin/activate
export CC=gcc-13 CXX=g++-13
export CUDA_HOME="$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13")')"
export PATH="$CUDA_HOME/bin:$PATH"

vllm serve openbmb/MiniCPM-OCR-1.0 \
  --served-model-name minicpm-ocr \
  --host 127.0.0.1 \
  --port 18499 \
  --max-model-len 40960 \
  --max-num-batched-tokens 40960 \
  --max-num-seqs 32 \
  --limit-mm-per-prompt '{"image":36,"video":0}' \
  --attention-backend FLASH_ATTN \
  --mm-encoder-attn-backend FLASH_ATTN \
  --mm-processor-kwargs '{"downsample_mode":"4x"}' \
  --gpu-memory-utilization 0.85 \
  --trust-request-mm-kwargs \
  --speculative-config '{
    "method": "dspark",
    "model": "openbmb/MiniCPM-OCR-1.0-DSpark",
    "num_speculative_tokens": 15,
    "enable_adaptive_verification": false
  }' \
  --default-chat-template-kwargs '{"enable_thinking":false}'

DSpark uses the matching MiniCPM-OCR-1.0-DSpark draft model. To use standard decoding, remove the entire --speculative-config argument and JSON value. Keep --trust-request-mm-kwargs for SDK requests.

Bash

curl -fsS http://127.0.0.1:18499/v1/models

Confirm minicpm-ocr appears in the model list before sending requests.

SGLang

SGLang serves MiniCPM-OCR through an OpenAI-compatible API. This section enables DSpark decoding with a matched target model and draft model. The SDK runs in a separate environment and connects over HTTP.

The commands target Linux x86_64, an NVIDIA GPU and Python 3.12. End-to-end SGLang deployment with these settings remains to be verified.

1. Prepare the source

MiniCPM-OCR 1.0 support is provided by PR #45799, which is open as of October 26, 2026. This guide uses the PR commit aaab6bfc3de5354b052f2e96d5e566d70b9f69a2, from branch model/MiniCPM-OCR-1.0.

Run all commands below from the root of the MiniCPM-OCR repository:

Bash

git clone https://github.com/sgl-project/sglang.git sources/sglang
git -C sources/sglang fetch origin pull/45799/head
git -C sources/sglang checkout aaab6bfc3de5354b052f2e96d5e566d70b9f69a2

If sources/sglang already exists, skip the clone and run the fetch and checkout commands.

2. Set up the SDK environment

Install the SDK and GPU PaddlePaddle in .minicpm-ocr as described in the Inference page. Reuse this environment if it is already installed for vLLM.

For CPU layout detection, follow the same steps but replace the paddlepaddle-gpu installation command with:

Bash

python -m pip install "paddlepaddle==3.2.1"

3. Install the service environment

The sglang profile in this repository's pyproject.toml installs sources/sglang/python and the CUDA 13 components. PyTorch 2.14.1, Transformers 5.19.0 and the other SGLang dependencies come from the source's own python/pyproject.toml. The installer uses a separate .venv-sglang environment because the vLLM profile requires a different PyTorch version.

Prepare the system build dependencies on Ubuntu 22.04. Run as root, or prefix APT commands with sudo:

Bash

apt-get update
apt-get install -y software-properties-common build-essential pkg-config libssl-dev curl ca-certificates protobuf-compiler
add-apt-repository -y ppa:ubuntu-toolchain-r/test
apt-get update
apt-get install -y gcc-13 g++-13 libdw-dev libelf-dev

Driver compatibility: CUDA 13 normally requires driver ≥580. For supported GPUs with an older driver, configure the compatibility library as described in the FAQ. Set its environment variables in the installation, service and GPU SDK terminals.

Install the service environment:

Bash

bash scripts/install.sh sglang
source .venv-sglang/bin/activate

The installer creates or reuses .venv-sglang, installs rustup if needed, and selects the Rust version pinned by the SGLang source (1.92 for this commit). It also prepares the CUDA compiler and library links. If the driver's libcuda.so.1 is outside /usr/lib/x86_64-linux-gnu, set OCR_LIBCUDA to its path before installation. When using the driver compatibility library, keep the value configured in the FAQ.

Check the environment:

Bash

uv pip check
"$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13/bin/nvcc")')" --version
python -c 'import sglang; print(sglang.__file__)'
python -c 'import torch, transformers; print("torch", torch.__version__, "CUDA", torch.version.cuda, "transformers", transformers.__version__); x = torch.ones(1, device="cuda"); print(torch.cuda.get_device_name(0), (x + 1).item())'

The SGLang path should point into sources/sglang/python/sglang; NVCC should report CUDA 13.0.

SGLang provides the model configuration and image processor for this PR. Its installation uses its own dependency list; the tutorial's Transformers adaptation checkout is used by vLLM and is not installed here.

4. Prepare the models

Download the SDK's PP-DocLayoutV3 layout model, or reuse the directory prepared for vLLM:

Bash

mkdir -p models
curl -fL -o ./models/PP-DocLayoutV3_infer.tar \
  https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayoutV3_infer.tar
tar xf ./models/PP-DocLayoutV3_infer.tar -C ./models

Download the MiniCPM-OCR-1.0 target model and its matched MiniCPM-OCR-1.0-DSpark draft model:

Bash

source .venv-sglang/bin/activate
hf download openbmb/MiniCPM-OCR-1.0 --local-dir ./models/MiniCPM-OCR-1.0
hf download openbmb/MiniCPM-OCR-1.0-DSpark --local-dir ./models/MiniCPM-OCR-1.0-DSpark

If the models are already present, reuse them. Set --model-path below to the DSpark target directory, --speculative-draft-model-path to its matched draft directory, and the SDK's layout.model_dir to the layout model directory.

5. Start the service

Open a service terminal at the repository root. On a single-GPU machine, finish the SDK run and stop the previous inference service before starting SGLang so that its GPU memory is available.

Older drivers: Set the compatibility-library environment variables in this terminal before starting the service, as described in the FAQ. Activating .venv-sglang does not restore variables from another terminal.

Bash

source .venv-sglang/bin/activate
export PATH="${CARGO_HOME:-$HOME/.cargo}/bin:$PATH"
export RUSTUP_TOOLCHAIN=1.92
export CC=gcc-13 CXX=g++-13
export CUDA_HOME="$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13")')"
export PATH="$CUDA_HOME/bin:$PATH"

CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server \
  --model-path ./models/MiniCPM-OCR-1.0 \
  --served-model-name minicpm-ocr \
  --host 127.0.0.1 --port 18499 \
  --context-length 40960 \
  --max-running-requests 32 \
  --mem-fraction-static 0.85 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path ./models/MiniCPM-OCR-1.0-DSpark \
  --default-chat-template-kwargs '{"enable_thinking":false}'

--speculative-algorithm DSPARK enables DSpark. SGLang reads the draft block size from the draft configuration and derives its verification window; do not copy vLLM's --speculative-config JSON into this command.

The source checkout can compile its Rust cache extension on first startup. Keep Cargo's directory in PATH and use Rust 1.92 in this terminal; the system Cargo may be older even after the installation script finishes. Reactivating the virtual environment resets PATH, so rerun all exports in this block after activation.

GPU and port: CUDA_VISIBLE_DEVICES=0 selects physical GPU 0. Change it to another GPU index when needed. If another service already uses port 18499, choose a free port here and use the same port in the SDK command below. Services sharing a GPU also share its available memory.

Concurrency: This SGLang configuration permits up to 32 running requests, including with DSpark enabled. The SDK example below sends at most 32 requests concurrently. Reduce both values if memory is insufficient; actual capacity also depends on image size and sequence length.

The chat-template setting disables thinking mode, as required by the adaptation PR. Check readiness in another terminal:

Bash

curl --noproxy 127.0.0.1 -f http://127.0.0.1:18499/v1/models

The returned model list should include minicpm-ocr.

6. Parse a document with the SDK

In a separate terminal at the repository root:

Bash

source .minicpm-ocr/bin/activate
minicpmvocr parse --config sdk/minicpmvocr/configs/minicpmvocr.yaml \
  --backend openai \
  --server-url http://127.0.0.1:18499 \
  --model minicpm-ocr \
  --set layout.model_dir="$PWD/models/PP-DocLayoutV3_infer" \
  --set layout.device=cuda:0 \
  --set pipeline.queue_mode=concurrent \
  --set engine.max_model_len=40960 \
  --set pipeline.max_workers=32 \
  --set pipeline.merge_cross_page_tables=false \
  --set pipeline.overlap_layout=true \
  --input doc.pdf --output doc.md

The input can be a PDF, PNG or JPG. PDFs are parsed in full; add --set input.limit=1 to check only the first page. The SDK model name must match the service's --served-model-name.

pipeline.overlap_layout=true lets layout detection and recognition run at the same time. pipeline.max_workers=32 limits concurrent recognition requests, rather than pages; adjust it for the service's capacity. pipeline.queue_mode applies when layout overlap is disabled. --backend openai selects the OpenAI-compatible client for SGLang.

CPU layout detection: Use the CPU PaddlePaddle environment configured above, activate .minicpm-ocr, and replace --set layout.device=cuda:0 with --set layout.device=cpu.

GPU selection: cuda:0 selects the first GPU visible to the SDK process. If that process uses CUDA_VISIBLE_DEVICES=3, its single visible GPU is addressed as cuda:0. New terminals using an older driver must also restore the compatibility-library environment variables from the FAQ before running GPU layout detection or the service.

Troubleshooting

Symptom	Check
Unknown `minicpm_ocr` model type or architecture	Confirm the source is at the pinned PR commit and was installed in `.venv-sglang`
CUDA driver error	Check driver compatibility and the compatibility-library environment variables in the FAQ
`nvcc` not found or NVCC linking files missing	Restore `CUDA_HOME` and `PATH` from step 5; rerun `bash scripts/install.sh sglang` if the compiler files are incomplete
NVCC reports an unknown `--compress-mode=size` option	Restore `CUDA_HOME` and `PATH` from the service command, then restart the service
Rust cache compilation fails with `edition2024` or Cargo exit code 101	Check the first compiler error. For `edition2024`, restore Cargo's `PATH` and `RUSTUP_TOOLCHAIN` from step 5; `cargo --version` should report 1.92.0
Out of GPU memory	Reduce `--max-running-requests`, SDK workers, or `--context-length`; keep `engine.max_model_len` consistent with the service
Thinking tags in the output	Keep `--default-chat-template-kwargs '{"enable_thinking":false}'` when starting the service

See the SGLang documentation for other hardware platforms and serving options.

llama.cpp

llama.cpp enables CPU and local GPU inference with GGUF models, ideal for resource-constrained environments or offline deployment.

Requirements

Hardware: CPU with AVX2, or NVIDIA/AMD GPU
RAM: ≥16 GB system memory
Storage: ~2.5 GB for the F16 model + vision projector

GGUF weights

Download from openbmb/MiniCPM-OCR-1.0-gguf:

File	Size	Description
`MiniCPM-OCR-1_0-F16.gguf`	1.4 GB	Language model (F16)
`mmproj-model-f16-4x.gguf`	859 MB	Vision projector (4x downsample)

Both files are required: the language model for --model, the projector for --mmproj.

Quantized versions (Q4_K_M, Q8_0) are available — see Quantization. To quantize them yourself:

Bash

./build/bin/llama-quantize MiniCPM-OCR-1_0-F16.gguf MiniCPM-OCR-1.0-Q8_0.gguf Q8_0

Installation

Build from source

Bash

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

# NVIDIA GPU
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server llama-quantize

# AMD GPU:   cmake -B build -DGGML_HIP=ON ...
# CPU only:  cmake -B build ...

Tips: on recent NVIDIA GPUs you can speed up the build by targeting one architecture, e.g. -DCMAKE_CUDA_ARCHITECTURES=120 (RTX 5090), 90 (H100), or 80 (A100). See the official build documentation for the full list of options.

Using pre-built binaries

Download from llama.cpp releases.

Serving

Start server

Bash

./build/bin/llama-server \
  --model ./MiniCPM-OCR-1_0-F16.gguf \
  --mmproj ./mmproj-model-f16-4x.gguf \
  --host 0.0.0.0 \
  --port 18499 \
  --ctx-size 40960 \
  --n-gpu-layers 999

Flags:

--mmproj FILE: path to the multimodal projector file — required for image input
--n-gpu-layers 999: offload all layers to GPU; 0 for CPU-only inference
--ctx-size: context window size

The official GGUF works out of the box — the embedded chat template already disables thinking mode, so no extra template flags are needed. (If you convert the GGUF weights yourself, start the server with --chat-template-kwargs '{"enable_thinking":false}'.)

Verify service

Bash

curl http://127.0.0.1:18499/health
# {"status":"ok"}

SDK configuration

Install the SDK and layout model as described in Inference, using .minicpm-ocr for the SDK environment. For CPU layout detection as shown below, replace the tutorial's paddlepaddle-gpu installation command with:

Bash

python -m pip install "paddlepaddle==3.2.1"

Then run from the root of the repository:

Bash

source .minicpm-ocr/bin/activate
export OCR_LAYOUT_MODEL_DIR="$PWD/PP-DocLayoutV3_infer"
export PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True
minicpmvocr parse --config sdk/minicpmvocr/configs/minicpmvocr.yaml \
  --backend llamacpp --server-url http://127.0.0.1:18499 \
  --model minicpm-ocr \
  --set layout.device=cpu \
  --set pipeline.queue_mode=serial \
  --set pipeline.max_workers=1 \
  --input doc.pdf --output doc.md

For GPU layout detection, install GPU PaddlePaddle as described in the inference page, activate .minicpm-ocr, and use --set layout.device=cuda:0 instead.

Concurrency: llama-server processes one request at a time by default. Keep pipeline.queue_mode=serial and pipeline.max_workers=1 in the SDK options.

Performance

Measured on a single RTX 5090 (32 GB) with the F16 weights:

Model + projector load: ~2.3 s
One table crop (1,312 prompt + 265 output tokens): ~1.0 s
VRAM: ~26 GB with --ctx-size 40960 — reduce the value on smaller GPUs
CPU-only (48 threads): one table image end-to-end (layout + recognition + assembly) in ~21 s

Quantization trade-offs

Quantization	Size	Table	Dense text / XML	Notes
F16	1.4 GB	baseline	baseline	highest fidelity
Q8_0	775 MB	lossless	≈lossless	recommended balance
Q4_K_M	505 MB	lossless	small degradation	closing-tag slashes may be dropped in dense code — prefer Q8_0 for code-heavy documents

Limitations

Quantization introduces small accuracy loss (~1–2% on benchmarks, most visible on dense code/XML at Q4_K_M)
No speculative decoding support (no DSpark)
Slower than vLLM on high-end GPUs

For more details, see the llama.cpp documentation.

Ollama

The upstream repository does not ship a standalone Ollama deployment guide. Ollama is supported as an SDK recognition backend via --backend ollama; the remaining options match the other backends:

Bash

minicpmvocr parse doc.pdf -o doc.md --device cuda:0 \
  --backend ollama \
  --server-url http://127.0.0.1:11434 \
  --model minicpm-ocr

Set the server URL and model name to match your local Ollama setup. See command-line options for the full parameter list.

FlagOS

The MiniCPM-OCR family supports deployment across multiple chip platforms through FlagOS, bringing document parsing and OCR confidence assessment to different compute platforms. Developers can deploy MiniCPM-OCR 1.0 on its own for OCR tasks or combine it with MiniCPM-OCR Jev to build a workflow of document parsing → confidence assessment → human review.

FlagOS overview

The Beijing Academy of Artificial Intelligence (BAAI), together with research institutions, chip manufacturers, system vendors, and organizations working on algorithms and software, established the FlagOS Open Source Community to support large-scale adoption of diverse AI chips.

FlagOS develops a unified, open-source system software stack for multiple AI chip architectures. Its core projects include an operator library, a unified AI compiler, parallel training and inference frameworks, and a unified communication library, connecting the model, system, and chip layers. Across supported platforms, developers can reuse model integrations and operator optimizations to develop once and migrate across chips, reducing the cost of moving between different chip software stacks.

Community website: flagos.io · Developer documentation: docs.flagos.io

FlagOS multi-chip support and downloads

FlagRelease is the FlagOS team's platform for automated model migration, adaptation, and release across AI chip architectures. Both models are available in BF16 for deployment across 12 chip and edge platforms: T-Head, NVIDIA, Huawei Ascend, Hygon, Kunlunxin, MetaX, Moore Threads, Tsingmicro, Iluvatar, Sunrise, M5, and CIX.

MiniCPM-OCR 1.0 (1.3B): An OCR-specialized model based on MiniCPM-V 4.6 for text recognition and document parsing, with support for photographed documents and complex tables.
MiniCPM-OCR Jev (2B): An independent model that assesses OCR results and provides calibrated confidence scores, helping human reviewers prioritize potentially incorrect OCR results.

The table below provides ModelScope and Hugging Face download links for the FlagOS-adapted deployment images of both models across supported chip platforms. See the corresponding model cards for model files, deployment code, and instructions for pulling images and starting containers.

Chip / Platform	MiniCPM-OCR 1.0	MiniCPM-OCR Jev
T-Head	ModelScope / Hugging Face	ModelScope / Hugging Face
NVIDIA	ModelScope / Hugging Face	ModelScope / Hugging Face
Huawei Ascend	ModelScope / Hugging Face	ModelScope / Hugging Face
Hygon	ModelScope / Hugging Face	ModelScope / Hugging Face
Kunlunxin	ModelScope / Hugging Face	ModelScope / Hugging Face
MetaX	ModelScope / Hugging Face	ModelScope / Hugging Face
Moore Threads	ModelScope / Hugging Face	ModelScope / Hugging Face
Tsingmicro	ModelScope / Hugging Face	ModelScope / Hugging Face
Iluvatar	ModelScope / Hugging Face	ModelScope / Hugging Face
Sunrise	ModelScope / Hugging Face	ModelScope / Hugging Face
ARM CPU (edge)	ModelScope / Hugging Face	ModelScope / Hugging Face

ARM edge deployments use the arm download links listed above, with BF16 versions for both models. Refer to the corresponding model cards for supported devices, runtime requirements, and deployment instructions.

FlagOS installation and usage

Option 1: Use a FlagRelease image (recommended)

Select the target chip platform and model from the table above. Choose MiniCPM-OCR 1.0 for document parsing; add MiniCPM-OCR Jev for the same platform if confidence assessment is required.
Follow the model card to prepare the host drivers and runtime, pull the corresponding image, and start a container with the required device mappings and mounts for the model and document image directories.
Run the OCR or MiniCPM-OCR Jev example provided by the model card in the corresponding environment. Use the platform-specific settings for model paths, devices, batch size, image preprocessing, and startup parameters.

To download the model files first, the following example uses the T-Head repositories on Hugging Face, matching the source installation examples below:

Bash

python -m pip install --upgrade huggingface_hub

# Download the OCR document parsing model
hf download FlagRelease/MiniCPM-OCR-1.0-BF16-zhenwu-FlagOS \
  --local-dir ./models/MiniCPM-OCR-1.0

# Optional: download the Jev confidence assessment model
hf download FlagRelease/MiniCPM-OCR-Jev-BF16-zhenwu-FlagOS \
  --local-dir ./models/MiniCPM-OCR-Jev

For another platform, replace the repository names with the corresponding entries in the table. Model files are also available through ModelScope. Inside a container, the model path should point to the mounted model directory.

Option 2: Install from source

MiniCPM-OCR 1.0: vLLM-plugin-FL

The following example sets up vLLM-plugin-FL for MiniCPM-OCR 1.0 on the T-Head platform, using vLLM v0.24.0, vllm-plugin-FL v0.3.0, and FlagGems v5.4.0. Before installation, prepare the chip drivers, runtime, PyTorch, and build dependencies as described in the model card.

Official vLLM-plugin-FL developer guide: GitHub / GitCode.

Bash

# 1. Install vLLM v0.24.0 (T-Head example)
git clone --branch v0.24.0 https://github.com/vllm-project/vllm.git
cd vllm
VLLM_TARGET_DEVICE=empty python -m pip install -v --no-build-isolation --no-deps .
cd ..

# 2. Install vllm-plugin-FL v0.3.0
git clone --branch v0.3.0 https://github.com/flagos-ai/vllm-plugin-FL.git
cd vllm-plugin-FL
python -m pip install --no-build-isolation -e .
cd ..

# 3. Install FlagGems v5.4.0
git clone --branch v5.4.0 https://github.com/flagos-ai/FlagGems.git
cd FlagGems
python -m pip install --no-build-isolation -e .
cd ..

Install the FlagTree unified compiler and the FlagCX unified communication library as required by your platform. See their project documentation and the corresponding model card for instructions. Refer to the model card for OCR model integration code, image preprocessing dependencies, and version and branch configurations for other platforms.

MiniCPM-OCR Jev: Torch-FL

MiniCPM-OCR Jev uses Torch-FL for multi-chip deployment. Torch-FL connects MiniCPM-OCR Jev's PyTorch and Transformers inference workflow to the chip backend, using FlagGems operators and the corresponding compiler to perform confidence assessment.

Official Torch-FL developer guide: GitHub / GitCode. SDK, PyTorch, and compiler dependencies vary by chip. Start with the installation guide to select your target platform.

The following example also targets T-Head. Run it in a separate Python environment prepared for MiniCPM-OCR Jev. First, follow the T-Head installation guide to prepare the PPU driver, SDK, compatible PyTorch build, build tools, and compiler dependencies. The example assumes the SDK is installed at /usr/local/PPU_SDK; adjust the path to match your installation.

Bash

mkdir -p minicpm-ocr-jev-src
cd minicpm-ocr-jev-src

# 1. Install Torch-FL (T-Head example)
git clone -b main https://github.com/flagos-ai/Torch-FL.git
cd Torch-FL
FLAGOS_ACCELERATOR=ppu \
  CUDA_HOME=/usr/local/PPU_SDK/CUDA_SDK \
  FLAGOS_SKIP_CUDA_ASSETS=1 \
  python -m pip install --no-build-isolation -v -e .
cd ..

# 2. Install the FlagGems operator library
git clone https://github.com/flagos-ai/FlagGems.git
cd FlagGems
python -m pip install --no-build-isolation -e .
cd ..

# 3. Install the FlagTree unified compiler if required by the platform
# See https://github.com/flagos-ai/FlagTree

# 4. Configure the Torch-FL runtime for T-Head
export FLAGOS_DISABLE_CUDA_ASSETS=1

For other platforms, use the build flags and runtime configuration from their installation guides. Follow the corresponding model card and platform guide for compatible Torch-FL, FlagGems, FlagTree, and any required FlagCX versions, as well as Transformers, MiniCPM-OCR Jev model code, and image rendering dependencies.

After installation, check whether a Torch-FL device is available in the same environment:

Python

import torch
import torch_fl

print("FlagOS available:", torch.flagos.is_available())
print("FlagOS device count:", torch.flagos.device_count())

If availability is True and the device count is greater than 0, continue with the MiniCPM-OCR Jev assessment workflow below. Configure model loading, device selection, and input preprocessing using the examples in the MiniCPM-OCR Jev model card.

Run document parsing and confidence assessment

1. Parse documents with MiniCPM-OCR 1.0

Load the OCR model for the target platform and supply document images using the provided example to obtain recognized text and document parsing results. Follow the model card for image resolution, page segmentation, prompts, and output formats. If MiniCPM-OCR Jev will be used afterward, retain each original page-region image and its corresponding OCR text, keeping them correctly matched.

2. Assess OCR results with MiniCPM-OCR Jev (optional)

Load the MiniCPM-OCR Jev model for the target platform in the configured Torch-FL environment. Following its example, prepare the original page-region image, an image rendered from the OCR text, and the corresponding recognized text. Run MiniCPM-OCR Jev's Transformers classification workflow through Torch-FL and apply the provided postprocessing to obtain calibrated confidence scores.

3. Prioritize human review

Validate a review threshold on representative documents from your application, then prioritize low-confidence content for human inspection. MiniCPM-OCR Jev can also be used after other OCR models, subject to validation of the input format and assessment quality on the target documents.

Troubleshooting

For an unsupported minicpm_ocr architecture, rerun the matching installation script. If compilation is killed, reduce MAX_JOBS. For out-of-memory errors, reduce context length and tune concurrency. Older drivers require the compatibility-library setup in the official FAQ.

MiniCPM-OCR Jev

The Docker deployment pins Chromium, CJK fonts and rendering dependencies to match the cloud rendering environment. A self-installed SDK renderer may shift absolute scores slightly; calibrate thresholds in the environment you deploy.

Requirements

The local tutorial requires an NVIDIA GPU with at least 12 GB VRAM, driver ≥535, and Python ≥3.10 for SDK deployment. Reserve about 5–9 GB for the forward environment, 1.5 GB for rendering and 2.5 GB for weights.

Docker

Download the BF16 weights on the quantization tab to a local directory first, then build from the repository:

Bash

git clone https://github.com/OpenBMB/MiniCPM-OCR.git
cd MiniCPM-OCR/scripts/minicpm-ocr-jev
docker build -f serve/Dockerfile -t minicpm-ocr-jev-1-0:latest .

docker run -d --name minicpm-ocr-jev --gpus '"device=0"' \
  -p 127.0.0.1:8030:8030 \
  -v /path/to/checkpoint:/weights:ro \
  minicpm-ocr-jev-1-0:latest 8030 /weights

curl -f http://127.0.0.1:8030/healthz

Replace /path/to/checkpoint with your downloaded model directory. After the health check succeeds, use the scoring example.

SDK and backends

The SDK separates the engine and service environments. It supports score-only deployment when rendered images are already available, or render-and-score deployment when only candidate text is supplied. The model service supports vLLM, SGLang, llama.cpp and Ollama. Follow the local deployment guide for the appropriate setup.

Topic	Link
Model Overview	MiniCPM-OCR
Inference	MiniCPM-OCR
Quantization	MiniCPM-OCR

Model Downloads​

MiniCPM-OCR 1.0​

Environment​

vLLM​

SGLang​

1. Prepare the source​

2. Set up the SDK environment​

3. Install the service environment​

4. Prepare the models​

5. Start the service​

6. Parse a document with the SDK​

Troubleshooting​

llama.cpp​

Requirements​

GGUF weights​

Installation​

Serving​

SDK configuration​

Performance​

Limitations​

Ollama​

FlagOS​

FlagOS overview​

FlagOS multi-chip support and downloads​

FlagOS installation and usage​

Run document parsing and confidence assessment​

Troubleshooting​

MiniCPM-OCR Jev​

Requirements​

Docker​

SDK and backends​

Related Resources​

Model Downloads

MiniCPM-OCR 1.0

Environment

vLLM

SGLang

1. Prepare the source

2. Set up the SDK environment

3. Install the service environment

4. Prepare the models

5. Start the service

6. Parse a document with the SDK

Troubleshooting

llama.cpp

Requirements

GGUF weights

Installation

Serving

SDK configuration

Performance

Limitations

Ollama

FlagOS

FlagOS overview

FlagOS multi-chip support and downloads

FlagOS installation and usage

Run document parsing and confidence assessment

Troubleshooting

MiniCPM-OCR Jev

Requirements

Docker

SDK and backends

Related Resources