VoxCPM2

VoxCPM Speech Series

Choose the deployment method that fits your needs: in-app integration, batch generation, an online service or on-device operation. This page covers the Python SDK, the CLI and the Web Demo, as well as ecosystem entry points such as Nano-vLLM-VoxCPM, vLLM-Omni and llama.cpp-omni; check the supported scope and runtime requirements of each backend separately.

VoxCPM2 deployment paths fall into two groups: the Python SDK, CLI and Web Demo provided by the OpenBMB core repository, and independently maintained ecosystem inference backends. Before choosing a backend, be clear about your goal: in-app integration, offline batch processing, a high-throughput API, or on-device operation without Python.

Deployment paths

Path	Maintainer and positioning	Use case	Interface
Python SDK / CLI	OpenBMB/VoxCPM core repository	Local integration, scripts, offline batch processing	Python / CLI
Web Demo	OpenBMB/VoxCPM core repository	Internal listening tests, parameter validation	Gradio
Nano-vLLM-VoxCPM	Independent community project, listed in the official ecosystem	High throughput and async requests on NVIDIA GPUs	Python; the project provides a FastAPI service
vLLM-Omni	Official vLLM omnimodal extension	OpenAI compatibility, multi-tenancy and batched concurrency	/v1/audio/speech
llama.cpp-omni	Independent community project, listed in the official ecosystem	On-device operation on CPU, Metal, CUDA, Vulkan	C++ CLI, GGUF

Installation commands, model formats and compatible versions for ecosystem projects are maintained by each project. Before production deployment, pin the commit, weight checksums and image dependencies, and run regression tests on the target hardware; don't rely solely on the reference commands on this page.

Python SDK integration

Bash

pip install voxcpm soundfile

Python

from pathlib import Path
from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained(
    "openbmb/VoxCPM2",
    load_denoiser=False,
)

def synthesize(text: str, output_path: str) -> str:
    wav = model.generate(
        text=text,
        cfg_value=2.0,
        inference_timesteps=10,
    )
    sf.write(output_path, wav, model.tts_model.sample_rate)
    return str(Path(output_path).resolve())

The service process should load the model once at startup; do not call from_pretrained repeatedly within each request. Concurrency, request queuing, timeouts, audio caching and the lifecycle of output files must be handled by the application layer.

Web Demo

Bash

git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
pip install -e .
python app.py --port 8808 --device auto

--device supports auto, cpu, mps, cuda and cuda:N. The Web Demo also downloads SenseVoice-Small for prompt-audio transcription. This entry point is suitable for trying things out and internal validation; it is not equivalent to a production service with authentication, rate limiting and monitoring.

Nano-vLLM-VoxCPM

Bash

pip install nano-vllm-voxcpm

Python

from nanovllm_voxcpm import VoxCPM
import numpy as np
import soundfile as sf

server = VoxCPM.from_pretrained(
    model="/path/to/VoxCPM2",
    devices=[0],
)
chunks = list(server.generate(target_text="你好，我来自 VoxCPM2。"))
sf.write("out.wav", np.concatenate(chunks), 48000)
server.stop()

The official README cites a reference RTF of about 0.13 on an RTX 4090, compared with about 0.30 for the standard PyTorch path. These numbers are reference values for specific hardware and implementations and do not reflect your text length, concurrency or parameter settings.

vLLM-Omni

Bash

uv pip install vllm==0.19.0 --torch-backend=auto
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
uv pip install -e .

vllm serve openbmb/VoxCPM2 --omni --port 8000

Bash

curl http://localhost:8000/v1/audio/speech   -H "Content-Type: application/json"   -d '{"model":"openbmb/VoxCPM2","input":"你好，欢迎使用 VoxCPM2。","voice":"default"}'   --output out.wav

vLLM-Omni is iterating rapidly. Before deploying, check its VoxCPM2 online serving example and installation docs to confirm the vLLM, PyTorch and device backend versions.

llama.cpp-omni on-device path

This path uses community-converted VoxCPM2 GGUF files and requires one BaseLM file and one Acoustic file.

Bash

git clone https://github.com/tc-mb/llama.cpp-omni.git
cd llama.cpp-omni
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target voxcpm2-cli -j

./build/bin/voxcpm2-cli   -t "你好，我正在端侧运行 VoxCPM2。"   -o output.wav   VoxCPM2-BaseLM-Q8_0.gguf   VoxCPM2-Acoustic-F16.gguf

CPU, Metal, CUDA and Vulkan are supported. The GGUF weights are maintained by the community and are not the original weights from the official OpenBMB model repository; conversion precision, quantization effects and file provenance need to be verified separately.

Deployment and integration via other community projects

ReadTheDocs also lists the following third-party project entry points. They are maintained independently by their respective communities and are not equivalent to the official OpenBMB Python implementation; before adopting one, confirm in the corresponding project the supported VoxCPM version, model file source, license, hardware requirements, feature coverage and precision impact.

Community project	ReadTheDocs entry
VoxCPM.cpp	Deployment docs
VoxCPM-ONNX	Deployment docs
VoxCPMANE	Deployment docs
MLX-Audio	Deployment docs
VoxCPM-RKNN2	Deployment docs
voxcpm_rs	Deployment docs
ComfyUI-VoxCPM	Integration docs
ComfyUI_RH_VoxCPM	Integration docs
ComfyUI-VoxCPMTTS	Integration docs
TTS WebUI	Integration docs

Production checklist

Dimension	Minimum checks
Versioning	Record the model repository revision, code commit, Python package lock file and container image digest.
Quality	Maintain fixed samples for Chinese and English, multilingual text, long text, numbers and symbols, cloning and control descriptions, and run regression tests across versions.
Performance	Record RTF, first-packet latency, total latency, concurrency, text length, sampling steps, peak VRAM and failure rate together.
Audio input	Check format, duration, noise, multiple speakers and consistency with the reference text; reject corrupted files.
Safety	Authentication, rate limiting, input length limits, file isolation, speaker authorization records and AI-generated content labeling.
Observability	Log request IDs, backend version, parameters, latency and error types; do not retain unauthorized voice samples in logs long term.

Official and ecosystem links

Project	Link
OpenBMB/VoxCPM	Core repository
Nano-vLLM-VoxCPM	Community inference backend
vLLM-Omni	vLLM omnimodal extension
llama.cpp-omni	On-device C++ backend