BitCPM4 is a ternary quantized model derived from MiniCPM through quantization-aware training, improving training and parameter efficiency.
Highlights
2 stage training — Improvements in training methods：Use a wind tunnel to search for hyperparameters on a small model; and adopt a two-stage training method - first train at high accuracy and then perform QAT, making full use of the trained high-accuracy model and significantly reducing the computing resources required in the QAT stage.
1.58bit Weight bit width — High parameter efficiency：Comparable performance to similar parametric full-precision models is achieved with a bit width of only 1.58 bits, demonstrating high parameter efficiency.
Benchmarks
Comprehensive benchmarks
The performance of BitCPM4 is comparable to other full-precision models of the same size. The comparison covers models such as Qwen3, Llama3.2, Gemma3 (BF16) and BitNet (three-value); "*" in the table refers to the value cited in the original report of the corresponding model.