VoxCPM2 supports three generation modes — voice design, voice cloning, and audio continuation. It can generate speech from a voice description in the text; generate speech with a similar timbre from a reference audio clip, with control instructions to adjust speed, emotion, or style; and continue audio when given both a reference clip and its transcript. The model supports streaming audio generation.
Highlights
Supports 30 languages, right out of the box：Based on 2 million+ hours of multilingual speech training, text can be synthesized by directly inputting text in any of the 30 supported languages, without the need for language tags. The average WER/CER is as low as 1.68% on an internal 30 language × 500 sample intelligibility benchmark (Gemini 3.1 Flash Lite evaluation), and can automatically infer appropriate rhythm and expressiveness based on text content.
Comprehensive voice design and cloning：Voice design: generate speech from a natural-language description (gender, age, pitch, emotion, speed), with no reference audio required;
Voice cloning: generate speech with a similar timbre from a short audio clip, with control instructions to adjust emotion, speed, and delivery;
Audio continuation: given both a reference clip and its transcript, continue from the reference audio while keeping timbre, rhythm, emotion, and style.
48kHz studio-grade sound quality and Real-time streaming：AudioVAE V2 asymmetric codec accepts 16kHz reference audio and directly outputs 48kHz studio-grade sound quality with a built-in super-resolution module without the need for an external upsampler.
RTF is as low as about 0.3 on NVIDIA RTX 4090, and can reach about 0.13 after Nano-vLLM / vLLM-Omni acceleration (supports PagedAttention and OpenAI compatible API).
The weights and code are released under the Apache-2.0 license, free for commercial use.
Technical highlights
Segmentizer-free diffusion autoregressive architecture：VoxCPM2 is built based on the tokenizer-free, diffusion autoregressive paradigm, runs entirely in the latent space of AudioVAE V2, and follows the LocEnc → TSLM → RALM → LocDiT four-stage pipeline to achieve rich expressiveness and natively output 48kHz audio.
Asymmetric audio codec and real-time streaming：AudioVAE V2 adopts an asymmetric codec design, 16kHz input directly outputs 48kHz, and has a built-in super-resolution module without the need for an external upsampler. Standard PyTorch implements RTF ~0.30 (RTX 4090), Nano-vLLM accelerates up to RTF ~0.13 and supports batch concurrency; the official vLLM-Omni full-modal service provides PagedAttention, continuous batch processing and OpenAI compatible APIs.
Voice design / voice cloning / audio continuation：Voice design: generate speech from a natural-language description. Voice cloning: reference audio plus control instructions to adjust emotion, speed, and delivery. Audio continuation: reference audio plus its transcript, continuing generation from the reference.