MiniCPM-RobotManip

MiniCPM-Robot Embodied Series

MiniCPM-RobotManip is a 1.5B general-purpose vision-language-action model for robot manipulation. It covers multiple downstream tasks with a single set of weights and retains historical visual context through streaming inference.

Model Download

Platform	Link
Hugging Face	openbmb/MiniCPM-RobotManip
GitHub	MiniCPM-RobotManip source code

Core Capabilities

• General-purpose manipulation: a single set of 1.5B weights covers a variety of downstream robot manipulation tasks.
• Streaming context: historical observations are continuously folded into the context, supporting up to about 1 minute of visual memory.
• Efficient visual encoding: inherits the visual token compression of MiniCPM-V 4.6, compressing each frame from 256 visual tokens to 64.
• Multi-view reasoning: supports different camera views, such as front and wrist cameras, provided as multiple image inputs.

Inputs and Outputs

Item	Description
Input	One or more images, a natural-language instruction, and optional robot state and embodiment ID
Output	Action chunks; the official single-sample script outputs a shape of (30, 80) by default

Quick Start

Follow this tutorial to set up the MiniCPM-RobotManip environment and run single-sample inference. The official test environment is Python 3.10, PyTorch 2.6.0, and CUDA 12.4.

Get the Code

Bash

git clone https://github.com/OpenBMB/MiniCPM-Robot.git
cd MiniCPM-Robot/MiniCPM-RobotManip

Create the Environment

Bash

conda env create -f environment.yml
conda activate MiniCPM-RobotManip

Prepare the Model Weights

Download the full model snapshot from Hugging Face and pass its path as the --checkpoint argument of the inference command.

Single-View Inference

Bash

python vla_infer.py \
  --image frame.jpg \
  --text "Pick up the red block." \
  --checkpoint ./checkpoint \
  --state-file state.npy \
  --embodiment-id 0 \
  --output action.npy

The robot state can be omitted, in which case the script uses a zero state by default. If --output is not passed, the action prediction is printed as JSON.

Multi-View Inference

Bash

python vla_infer.py \
  --image cam_front.jpg \
  --image cam_wrist.jpg \
  --text "Pick up the red block." \
  --checkpoint ./checkpoint

Output

The official single-sample inference script returns an action chunk with a default shape of (30, 80). Before connecting it to a specific robot, you need to implement action mapping and safety constraints based on the embodiment, state dimensions, and control interface.

Related resources

Topic	Link
Deployment	MiniCPM-RobotManip Deployment Guide
Evaluation and reproduction	MiniCPM-RobotManip Evaluation & Reproduction
Official source code	MiniCPM-RobotManip
Model weights	openbmb/MiniCPM-RobotManip