VoxCPM1.5

VoxCPM Speech Series

Prepare Chinese and English speech data with accurate transcriptions, and start customizing a voice with LoRA or full-parameter fine-tuning. Follow the guide through data validation, training configuration and checkpoint verification, and use text outside the training set to check audio quality and generalization.

VoxCPM1.5 supports both LoRA and full-parameter fine-tuning. Both use the same JSONL manifest and training script; for most speaker customization tasks, start with LoRA. The official fine-tuning docs give reference VRAM figures of about 12 GB for LoRA and about 24 GB for full-parameter fine-tuning.

Set Up the Environment

Bash

git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
pip install -e .

python -c "from huggingface_hub import snapshot_download; snapshot_download('openbmb/VoxCPM1.5', local_dir='./pretrained_models/VoxCPM1.5')"

Data Format

JSON

{"audio":"data/audio_001.wav","text":"第一条音频的逐字转写。","duration":4.5}
{"audio":"data/audio_002.wav","text":"第二条音频的逐字转写。","duration":7.2}

Item	VoxCPM1.5 Requirement
Manifest	JSONL, with audio and text on each line.
Clip duration	The official docs recommend 3–30 seconds.
Audio quality	Single speaker, low noise, low reverb, consistent volume.
Transcription	Must match word for word; don't put prompts or spoken-out readings of punctuation that wasn't actually voiced into the text.
Sample rate	The config uses sample_rate: 44100, which must match the 1.5 AudioVAE configuration.

Bash

voxcpm validate   --manifest train.jsonl   --sample-rate 44100

LoRA Fine-tuning

Bash

cp conf/voxcpm_v1.5/voxcpm_finetune_lora.yaml conf/my_voxcpm15_lora.yaml

YAML

pretrained_path: ./pretrained_models/VoxCPM1.5
train_manifest: ./train.jsonl
val_manifest: ./val.jsonl
sample_rate: 44100
batch_size: 16
grad_accum_steps: 1
learning_rate: 0.0001
max_steps: 2000
save_interval: 1000
save_path: ./checkpoints/voxcpm15_lora
tensorboard: ./logs/voxcpm15_lora
lora:
  enable_lm: true
  enable_dit: true
  enable_proj: false
  r: 8
  alpha: 16
  dropout: 0.0

Bash

python scripts/train_voxcpm_finetune.py   --config_path conf/my_voxcpm15_lora.yaml

The parameters above come from the repository's 1.5 example config. They are a starting baseline, not the optimal values for every dataset. If you run out of VRAM, first lower batch_size, then use grad_accum_steps to keep the effective batch size.

Full-parameter Fine-tuning

Bash

cp conf/voxcpm_v1.5/voxcpm_finetune_all.yaml conf/my_voxcpm15_full.yaml
python scripts/train_voxcpm_finetune.py   --config_path conf/my_voxcpm15_full.yaml

The full-parameter config has no lora section, and the example learning rate is 1e-5. It overfits more easily on small datasets, so you must keep a validation set and save multiple checkpoints.

Multi-GPU Training

Bash

CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2   scripts/train_voxcpm_finetune.py   --config_path conf/my_voxcpm15_lora.yaml

Verify the Model

Bash

# LoRA
python scripts/test_voxcpm_lora_infer.py   --lora_ckpt checkpoints/voxcpm15_lora/step_0001000   --text "这是训练集以外的验证文本。"   --output voxcpm15_lora.wav

# 全参数
python scripts/test_voxcpm_ft_infer.py   --ckpt_dir checkpoints/voxcpm15_full/step_0001000   --text "这是训练集以外的验证文本。"   --output voxcpm15_full.wav

Evaluation and Troubleshooting

Problem	What to Check
Timbre is similar but words are dropped	Overfitting or inconsistent transcriptions; compare earlier checkpoints on unseen text.
Audio quality degrades	Check the training audio's sample rate, noise and reverb, and whether denoising was mistakenly enabled.
Generation doesn't stop	Check loss/stop, clip durations and anomalous samples.
Results aren't reproducible	Record the model revision, repository commit, config, data manifest hash and random seed.

The LoRA WebUI can be launched with python lora_ft_webui.py; for formal experiments you should still save the YAML and data versions. For full details, see the official fine-tuning docs。