Create Voices from Nothing and Clone in Seconds: The Complete VoxCPM2 Deployment and Fine-Tuning Guide
Recently, VoxCPM2 has received widespread attention and praise from the community 👏! As a new version with 2 billion parameters, it unlocks a range of impressive voice capabilities for community partners:
🎨 Voice design: describe a voice in words (e.g., "young woman, gentle and sweet") and create an entirely new voice from nothing.
🎭 Controllable voice cloning: with just 5–6 seconds of reference audio, perfectly reproduce a timbre, and freely control style such as speaking rate and emotion.
✨ Ultimate cloning (audio continuation): reproduce every vocal detail for the highest-fidelity clone.
As the model's influence in the community has grown, we've received a flood of messages from developers: "I want to run it locally — is there a hand-holding tutorial?" "How do I fine-tune my own voice?"
Consider it done! Today we've prepared a complete local deployment and fine-tuning guide for absolute beginners. Whether you're a newcomer who just wants to try the Web Demo or a geek putting the model into production, this tutorial walks you through the entire VoxCPM2 workflow, step by step.
Get your GPU ready — let's begin! 👇
➤ Reference deployment environment
1. OS: Ubuntu 22.04 / 24.04 (Windows WSL2 / macOS can also be used as a reference)
2. GPU
NVIDIA GPU with ≥ 8 GB VRAM (e.g., RTX 3080, RTX 4070 or above);
Apple Silicon (M1/M2/M3/M4) can run via the MPS backend; unified memory ≥ 16 GB recommended
3. Drivers: NVIDIA GPUs require NVIDIA Driver ≥ 535 and CUDA ≥ 12.0; Apple Silicon needs no additional drivers
4. Python: 3.10–3.12 (3.13 not yet supported)
5. Reference docs: the official VoxCPM documentation (https://voxcpm.readthedocs.io/zh-cn/latest/)
About VoxCPM2: VoxCPM2 is a 2-billion-parameter text-to-speech (TTS) model supporting 30 languages, 9 Chinese dialects, voice design (create new voices from text descriptions), controllable voice cloning, and 48 kHz high-fidelity audio output. It is released under the Apache-2.0 open-source license.
Hardware requirements at a glance
🍎 Apple Silicon (M1/M2/M3/M4) users: VoxCPM automatically detects and uses the MPS backend, but it has not been specifically adapted for MPS; torch.compile acceleration is unavailable, some operators may fall back to CPU, and it is slower than on NVIDIA GPUs. For a better Apple chip experience, we recommend the community project VoxCPMANE (https://github.com/0seba/VoxCPMANE, Apple Neural Engine acceleration):
Model download links:
➤ VoxCPM2 (recommended)
HuggingFace:
🔗 https://huggingface.co/openbmb/VoxCPM2
ModelScope:
🔗 https://modelscope.cn/models/OpenBMB/VoxCPM2/summary
➤ VoxCPM1.5
HuggingFace:
🔗 https://huggingface.co/openbmb/VoxCPM1.5
ModelScope:
🔗 https://modelscope.cn/models/OpenBMB/VoxCPM1.5/summary
➤ VoxCPM-0.5B
HuggingFace:
🔗 https://huggingface.co/openbmb/VoxCPM-0.5B
ModelScope:
🔗 https://modelscope.cn/models/OpenBMB/VoxCPM-0.5B/summary
💡 Users in China are advised to download from ModelScope for faster speeds. After installing with pip install voxcpm, the model downloads automatically from Hugging Face on first run; to download manually, get it from the links above and place it in a local directory.
💡 If you don't have a dedicated GPU or Apple Silicon, you can also run on CPU, but it will be much slower.
Minimum requirements: before you start deploying, confirm that your computer meets the minimum requirements of either option below:
❓ How do I know which option to choose?
Windows / Linux users: If you have an NVIDIA discrete GPU (e.g., GTX/RTX series), go with Option A.
Mac users: If your Mac was purchased after late 2020 (M1 chip or later), go with Option B.
Neither? You can still run on CPU, but it will be slow. We suggest trying the online Demo first.
Environment setup: 2.1 Check GPU and drivers
Step 1: Check the NVIDIA driver version
nvidia-smi
You can see CUDA Version in the upper right of the output; make sure it is ≥ 12.0. If the command is not found or the version is too low, install or update the NVIDIA driver first (https://www.nvidia.com/en-us/drivers/).
Step 2: Check VRAM size
Also in the nvidia-smi output, look at the GPU's Memory column. VoxCPM2 needs ≥ 8 GB of VRAM.
2.2 Install Python
We recommend Miniconda (https://docs.conda.io/en/latest/miniconda.html) for managing Python environments:
# Create a Python 3.10 virtual environment
conda create -n voxcpm python=3.10 -y
# Activate the environment (all following steps run inside it)
conda activate voxcpm
If you don't use conda, you can create a virtual environment with python3 -m venv; just make sure the Python version is 3.10–3.12 (3.13 is not yet supported).
Step 3: Verify the Python version
python --version
The output should be Python 3.10.x, 3.11.x, or 3.12.x.
2.3 Install PyTorch
Note: pip install voxcpm installs PyTorch automatically, but it may install the CPU build by default. If you use an NVIDIA GPU, we recommend installing a PyTorch build that matches your CUDA version first, then installing VoxCPM, to ensure GPU acceleration works properly. Apple Silicon and CPU users can skip this step and go straight to Section 3 to install VoxCPM.
Go to the official PyTorch install page (https://pytorch.org/get-started/locally/), choose the command matching your CUDA version, for example:
# CUDA 12.4 example (adjust for your actual CUDA version)
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu124
Step 4: Verify PyTorch and CUDA
python -c "import torch; print(f'PyTorch {torch.__version__}, CUDA {torch.version.cuda}, GPU available: {torch.cuda.is_available()}')"
The output should show GPU available: True.
Install VoxCPM
There are two installation methods; choose either one.
Option A: pip install (recommended, simplest)
pip install voxcpm
Step 5: Verify the installation
python -c "from voxcpm import VoxCPM; print('VoxCPM installed successfully!')"
Option B: Install from source (needed for the Web Demo or fine-tuning)
# Clone the repository
git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
# Install in editable mode (installs all dependencies)
pip install -e .
🎁 Note: the project manages dependencies with pyproject.toml, not requirements.txt. pip install -e . installs all required dependencies automatically.
Launch the Web Demo (the fastest way to try it)
The Web Demo provides a visual web interface — try every feature without writing any code.
4.1 Launch from source
Make sure you have cloned the project as described in Option B:
cd VoxCPM
# Start the Web Demo
python app.py
On startup it automatically downloads the model weights from HuggingFace (several GB; a stable network is needed on first launch), and you'll see output like this in the terminal:
Running on local URL: http://0.0.0.0:8808
Step 6: Open it in your browser — go to http://localhost:8808.
4.2 Optional parameters
# Use a local model directory (skip auto-download)
python app.py --model-id /path/to/your/VoxCPM2
# Specify a port
python app.py --port 9000
You can also specify the model directory via an environment variable:
export VOXCPM_MODEL_DIR=/path/to/your/VoxCPM2
python app.py
4.3 Try the online Demo on Huggingface & Modelscope — no installation required
Online Demo, no install (international): https://huggingface.co/spaces/openbmb/VoxCPM-Demo
Online Demo, no install (China): https://voxcpm.modelbest.cn/
In addition, so that more beginners interested in AI can actually get hands-on with VoxCPM2, the OpenBMB "Little Cannon" open-source ambassador has open-sourced a local app 👍 — just download and double-click to use. Community members are welcome to download and try it! ❤️
➤ Download link
🔗 https://github.com/ZMXJJ/Voca/releases
Python API usage: If you want to call VoxCPM2 from your own Python programs, here are the core usage patterns. 5.1 Basic text-to-speech
from voxcpm import VoxCPM
import soundfile as sf
# Load the model (downloads from HuggingFace on first run)
model = VoxCPM.from_pretrained(
"openbmb/VoxCPM2",
load_denoiser=False,
)
# Generate speech
wav = model.generate(
text="Welcome to VoxCPM2, a powerful speech synthesis system.",
cfg_value=2.0,
inference_timesteps=10,
)
# Save as a WAV file
sf.write("demo.wav", wav, model.tts_model.sample_rate)
print("Saved: demo.wav")
💡 Parameter notes:
cfg_value: guidance strength; higher values follow the text more closely; recommended range 1.5–3.0
inference_timesteps: number of diffusion steps; higher values improve audio quality but slow generation; recommended range 10–32
5.2 Voice design (no reference audio needed)
Use a natural-language description to "design" a brand-new voice; the format is a description wrapped in parentheses at the start of the text:
wav = model.generate(
text="(Young female, gentle and sweet voice) Hello, welcome to the world of VoxCPM2!",
cfg_value=2.0,
inference_timesteps=10,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)
5.3 Controllable voice cloning
Upload a piece of reference audio; after cloning the timbre, you can also control speaking rate, emotion, and more with instructions:
# Basic cloning
wav = model.generate(
text="This is a cloned voice.",
reference_wav_path="/path/to/voice.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)
# Cloning with style control
wav = model.generate(
text="(Slightly faster, cheerful tone) This is a cloned voice with style control.",
reference_wav_path="/path/to/voice.wav",
cfg_value=2.0,
inference_timesteps=10,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)
🎁 Reference audio requirements: WAV format, 16 kHz sample rate, mono, 5–15 seconds long, and free of background noise.
5.4 Ultimate cloning (audio continuation)
Provide both reference audio and its text transcript for the highest-fidelity clone:
wav = model.generate(
text="This demonstrates ultimate cloning.",
prompt_wav_path="/path/to/voice.wav",
prompt_text="Transcript of the reference audio.",
reference_wav_path="/path/to/voice.wav",
)
sf.write("ultimate_clone.wav", wav, model.tts_model.sample_rate)
5.5 Streaming generation
import numpy as np
chunks = []
for chunk in model.generate_streaming(
text="VoxCPM2 supports streaming speech generation, playing while it generates!",
):
chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, model.tts_model.sample_rate)
CLI usage: After installing VoxCPM, you can use the voxcpm command directly in the terminal. Voice design:
voxcpm design \
--text "VoxCPM2 brings studio-grade multilingual speech synthesis." \
--output out.wav
Voice design with a description:
voxcpm design \
--text "VoxCPM2 brings studio-grade multilingual speech synthesis." \
--control "Young female voice, warm and soft, with a slight smile" \
--output out.wav
Voice cloning:
voxcpm clone \
--text "This is a speech cloning demo." \
--reference-audio /path/to/voice.wav \
--output out.wav
Ultimate cloning:
voxcpm clone \
--text "This is a speech cloning demo." \
--prompt-audio /path/to/voice.wav \
--prompt-text "Transcript of the reference audio" \
--reference-audio /path/to/voice.wav \
--output out.wav
Batch processing:
voxcpm batch --input examples/input.txt --output-dir outs
View help:
voxcpm --help
voxcpm design --help
voxcpm clone --help
💡 Default parameters: --cfg-value defaults to 2.0, --inference-timesteps defaults to 10. The model downloads openbmb/VoxCPM2 from HuggingFace by default. To use a local model, add --model-path /path/to/VoxCPM2.
LoRA fine-tuning primer
With just 5–10 minutes of audio data, you can adapt the model to a specific speaker, language, or domain.
7.1 Prepare training data
Training data is a JSONL file, one record per line:
{"audio": "data/audio1.wav", "text": "Transcript for the audio."}
{"audio": "data/audio2.wav", "text": "Another transcript.", "duration": 3.5}
The project includes a sample file for reference: examples/train_data_example.jsonl
7.2 Edit the configuration file
For LoRA fine-tuning, edit conf/voxcpm_v2/voxcpm_finetune_lora.yaml: minimal configuration (only three items need changing):
pretrained_path: /path/to/VoxCPM2/ # change to your model directory
train_manifest: /path/to/train.jsonl # change to your training data path
save_path: /path/to/checkpoints/my_lora # change to where you want to save checkpoints
Full configuration reference:
pretrained_path: /path/to/VoxCPM2/
train_manifest: /path/to/train.jsonl
val_manifest: null
sample_rate: 48000
batch_size: 2
grad_accum_steps: 8
num_workers: 8
num_iters: 1000
log_interval: 10
valid_interval: 500
save_interval: 500
learning_rate: 0.0001
weight_decay: 0.01
warmup_steps: 100
max_steps: 1000
max_batch_tokens: 8192
save_path: /path/to/checkpoints/finetune_lora
tensorboard: /path/to/logs/finetune_lora
lambdas:
loss/diff: 1.0
loss/stop: 1.0
lora:
enable_lm: true
enable_dit: true
enable_proj: false
r: 32
alpha: 32
dropout: 0.0
7.3 Start training
python scripts/train_voxcpm_finetune.py \
--config_path conf/voxcpm_v2/voxcpm_finetune_lora.yaml
7.4 Train and infer with the WebUI
If you prefer a graphical interface:
python lora_ft_webui.py
Open http://localhost:7860 in your browser to configure data, start training, and test inference on the web page.
For a detailed fine-tuning guide, see the official fine-tuning documentation (https://voxcpm.readthedocs.io/zh-cn/latest/finetuning/finetune.html).
Production deployment (Nano-vLLM acceleration)
If you need to serve TTS at high throughput in production, we recommend the Nano-vLLM-VoxCPM (https://github.com/a710128/nanovllm-voxcpm) inference engine:
pip install nano-vllm-voxcpm
from nanovllm_voxcpm import VoxCPM
import numpy as np, soundfile as sf
# Start the service (specify GPU devices)
server = VoxCPM.from_pretrained(model="/path/to/VoxCPM2", devices=[0])
# Generate speech
chunks = list(server.generate(target_text="Hello from VoxCPM!"))
sf.write("out.wav", np.concatenate(chunks), 48000)
# Stop the service
server.stop
On an RTX 4090, RTF is as low as ~0.13 (about ~0.3 for the standard PyTorch implementation), with support for batched concurrent requests and a FastAPI HTTP service. See the Nano-vLLM-VoxCPM repository (https://github.com/a710128/nanovllm-voxcpm).
FAQ and troubleshooting
Q1: nvidia-smi shows a CUDA version that doesn't match PyTorch
nvidia-smi shows the highest CUDA version supported by the driver, while PyTorch uses the CUDA version it was compiled with. The two don't need to match exactly — the driver version only needs to be ≥ the CUDA version PyTorch requires. To check:
python -c "import torch; print(torch.version.cuda)"
If the version is a problem, go to the PyTorch install page (https://pytorch.org/get-started/locally/), choose a build matching your CUDA driver, and reinstall.
Q2: GPU out of memory
Try a smaller model: VoxCPM-0.5B needs only ~5 GB of VRAM
Use half precision when loading the model: torch_dtype=torch.float16
Reduce the generated text length or lower max_length
Q3: Model download is too slow or fails
HuggingFace model downloads can be slow in China. You can:
# Option 1: set the HuggingFace mirror
export HF_ENDPOINT=https://hf-mirror.com
# Option 2: download the model manually from ModelScope
# Visit https://modelscope.cn/models/OpenBMB/VoxCPM2, download and extract it to a local directory
# Then point --model-id to the local path
python app.py --model-id /path/to/downloaded/VoxCPM2
Q4: What are the reference audio requirements?
Q5: How do I tune generation quality if it's not good enough?
Increase cfg_value (recommended 1.5–3.0); higher values follow the text more closely
Increase inference_timesteps (recommended 10–32); better audio quality but slower
For cloning, make sure the reference audio is clear and noise-free
Q6: Does it support macOS / Apple Silicon? VoxCPM supports CPU inference (but it is slow); Apple Silicon users can try the community project VoxCPMANE (https://github.com/0seba/VoxCPMANE) for Apple Neural Engine acceleration. Q7: How do I specify which GPU to use?
# Use only GPU 0
CUDA_VISIBLE_DEVICES=0 python app.py
# Use GPU 1
CUDA_VISIBLE_DEVICES=1 python app.py
🧾 Environment variable quick reference
📚 For more usage and advanced features, see:
➤ Quick start guide
🔗 https://voxcpm.readthedocs.io/zh-cn/latest/quickstart.html
➤ Usage guide
🔗 https://voxcpm.readthedocs.io/zh-cn/latest/usage_guide.html
➤ CookBook
🔗 https://voxcpm.readthedocs.io/zh-cn/latest/cookbook.html
From environment setup to generating your first sentence of speech, and on to LoRA fine-tuning with just 5–10 minutes of audio, you should now have VoxCPM2 up and running locally!
For developers with high-concurrency service needs, don't forget to try the Nano-vLLM-VoxCPM inference engine mentioned above — it will give your production environment an extra edge. If you run into any problems during deployment, or produce a stunning "heavenly voice," community members are welcome to join the official VoxCPM2 Feishu group to share and give us feedback!
🌟 If you like VoxCPM2, come give us a Like on Hugging Face 💗 to show your support!
➤ Hugging Face
🔗https://huggingface.openbmb.com/model/openbmb/VoxCPM2
➤ GitHub
🔗https://github.com/OpenBMB/VoxCPM/
➤ ModelScope
🔗 https://modelscope.cn/models/OpenBMB/VoxCPM2
➤ GitCode
🔗 https://ai.gitcode.com/OpenBMB/VoxCPM2
➤ Modelers
🔗 https://modelers.cn/models/OpenBMB/VoxCPM2