MiniCPM-RobotTrack

MiniCPM-Robot Embodied Series

MiniCPM-RobotTrack is a 0.9B on-device vision-language-action model for embodied target tracking. It covers static, dynamic and adversarial targets and provides a real-robot deployment workflow for the Unitree Go2 EDU.

Model downloads

Platform	Link
Hugging Face	openbmb/MiniCPM-RobotTrack
GitHub	MiniCPM-RobotTrack source code

Core capabilities

• Multi-type target tracking: Covers target tracking in static, dynamic and adversarial scenarios.
• On-device execution: Achieves 5+ FPS on the native onboard compute of the Unitree Go2 EDU, with end-to-end latency of about 180 ms.
• Natural-language interaction: Performs tracking from visual input and a natural-language description of the target, with no need to chain separate perception, planning and control models.
• Continuous data improvement: Combines automated quality checks, human review and DAgger iterations to improve performance in long-tail scenarios.

Supported scenarios

Scenario	Description
STT	Static target tracking
DT	Dynamic target tracking
AT	Adversarial target tracking

Safety note: The first run of a real-robot deployment must stay in dry-run mode. Verify the camera, model, latency, speed limits and emergency-stop chain with the robot on a stand, and only then enable real motion control following the official documentation.

Quick Start

This guide walks you through creating the MiniCPM-RobotTrack simulation environment and preparing the basic data. The official simulation environment has been tested with Python 3.9, Habitat-Sim 0.3.1, Bullet, PyTorch 2.4.1 and CUDA 12.1.

Get the code

Bash

git clone https://github.com/OpenBMB/MiniCPM-Robot.git
cd MiniCPM-Robot/MiniCPM-RobotTrack

Create the environment

Bash

conda env create -f environment.yml
conda activate MiniCPM-RobotTrack

Prepare simulation assets

Follow the asset instructions from Habitat-Sim and TrackVLA to download the HM3D, MP3D, humanoid and robot assets, and keep the original directory structure.

Plain Text

data/
├── datasets/
├── scene_datasets/
├── humanoids/
└── robots/

Prepare the model

Download the full model snapshot from Hugging Face. Evaluation also requires the DINOv3 ViT-S/16 and SigLIP So400m vision models.

Related resources

Topic	Link
Deployment	MiniCPM-RobotTrack Deployment Guide
Training trajectories and fine-tuning	MiniCPM-RobotTrack Fine-tuning Guide
Evaluation and reproduction	MiniCPM-RobotTrack Evaluation & Reproduction
Official source code	MiniCPM-RobotTrack
Model weights	openbmb/MiniCPM-RobotTrack