VoxCPM-0.5B

VoxCPM Speech Series

Reuse the training configuration of the first-generation model, prepare speech data with verbatim transcripts, and run LoRA or full-parameter fine-tuning. Compare timbre, intelligibility and stability on out-of-training text, and preserve the base model's Chinese and English speech generation capability.

VoxCPM-0.5B supports LoRA and full-parameter fine-tuning; the corresponding config directory in the repository is conf/voxcpm_v1/. Use the version directory provided in the repository directly — do not derive the config path yourself from the public model name.

Setting Up the Environment

Bash

git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
pip install -e .

python -c "from huggingface_hub import snapshot_download; snapshot_download('openbmb/VoxCPM-0.5B', local_dir='./pretrained_models/VoxCPM-0.5B')"

Training Data

JSON

{"audio":"data/audio_001.wav","text":"第一条音频的逐字转写。","duration":4.0}
{"audio":"data/audio_002.wav","text":"第二条音频的逐字转写。","duration":6.5}

Check	Requirement
Format	JSONL, with audio and text on each line.
Clips	The official general docs recommend 3–30 seconds; remove clips shorter than 1 second and abnormally long clips.
Quality	Single speaker, low noise, low reverb, consistent volume; text matches the audio verbatim.
Sample rate	The 0.5B config uses sample_rate: 16000.
Validation set	Hold out text and audio that are not used in training, rather than only listening to training-set samples.

Bash

voxcpm validate   --manifest train.jsonl   --sample-rate 16000

LoRA Fine-tuning

Bash

cp conf/voxcpm_v1/voxcpm_finetune_lora.yaml conf/my_voxcpm05_lora.yaml

YAML

pretrained_path: ./pretrained_models/VoxCPM-0.5B
train_manifest: ./train.jsonl
val_manifest: ./val.jsonl
sample_rate: 16000
batch_size: 16
grad_accum_steps: 1
learning_rate: 0.0001
max_steps: 2000
save_interval: 1000
save_path: ./checkpoints/voxcpm05_lora
tensorboard: ./logs/voxcpm05_lora
lora:
  enable_lm: true
  enable_dit: true
  enable_proj: false
  r: 8
  alpha: 16
  dropout: 0.0

Bash

python scripts/train_voxcpm_finetune.py   --config_path conf/my_voxcpm05_lora.yaml

The example parameters in the repository are a runnable baseline. The official fine-tuning page does not provide a separate training VRAM table for 0.5B, so measure on your target hardware starting from a small batch, rather than extrapolating from the numbers for 1.5 or 2.

Full-parameter Fine-tuning

Bash

cp conf/voxcpm_v1/voxcpm_finetune_all.yaml conf/my_voxcpm05_full.yaml
python scripts/train_voxcpm_finetune.py   --config_path conf/my_voxcpm05_full.yaml

The full-parameter example uses a learning rate of 1e-5 and has no lora section. On small datasets, full-parameter training is more likely to damage the base text-to-speech capability, so save checkpoints more frequently and use the validation set for early stopping.

Validating Checkpoints

Bash

# LoRA
python scripts/test_voxcpm_lora_infer.py   --lora_ckpt checkpoints/voxcpm05_lora/step_0001000   --text "这是一条未参与训练的验证文本。"   --output voxcpm05_lora.wav

# 全参数
python scripts/test_voxcpm_ft_infer.py   --ckpt_dir checkpoints/voxcpm05_full/step_0001000   --text "这是一条未参与训练的验证文本。"   --output voxcpm05_full.wav

Acceptance Criteria

Dimension	Check
Timbre	Compare against target-speaker samples via blind listening or in a fixed order.
Intelligibility	Use out-of-training Chinese and English text to check for dropped words, swallowed syllables, mispronunciations and failure to stop.
Generalization	Cover short sentences, long sentences, numbers, English abbreviations and unseen content.
Regression	Compare with the base model to confirm personalization hasn't broken basic Chinese and English TTS.
Reproducibility	Save the data manifest hash, model revision, code commit, YAML and random seed.

The LoRA WebUI can be started with python lora_ft_webui.py. For complete training behavior and FAQs, refer to the official fine-tuning documentation.