ModelBest and Tsinghua open-source BitCPM-CANN, China’s first 1.58-bit on-device model trained on Ascend
From May 25 to 29, ModelBest and OpenBMB are co-hosting the On-Device Large Model Open Source Week, unveiling one killer capability of on-device models each day. The peak of on-device models is not just the tip of the iceberg, but the entire iceberg. Today brings the first installment: BitCPM-CANN, our result in low-bit large-model training.
ModelBest, together with Tsinghua University and the OpenBMB open-source community, with support from the Tsinghua University Kunpeng-Ascend Science and Education Innovation Excellence Center, today officially releases and open-sources BitCPM-CANN, its latest result in low-bit large-model training.
China's first open-source ternary (1.58-bit) large model trained end to end entirely on a domestic compute platform (Huawei Ascend) made its first technical appearance at the Huawei Kunpeng & Ascend Developer Conference (KADC 2026) on May 23. Today its full model series is open to everyone.
From quantization kernels and training algorithms to the full-pipeline framework, BitCPM-CANN was built natively on Huawei Ascend. It comes in four sizes — 0.5B, 1B, 3B, and 8B — and was evaluated item by item against the full-precision MiniCPM4 family at the same sizes, with strong results.
Compared with conventional BF16 precision, BitCPM-CANN frees up roughly 6x the VRAM during inference while retaining 90%–97.2% of model capability.
Phone memory determines how many "experts" can run at once, and therefore the total model scale a device can hold.
The same device memory can now hold far more model capability; conversely, the same model capability runs on the same device with only one-sixth the memory — which means a 60B model may run on a phone in the future, greatly raising what phones can do.
"2-bit" has been a hot research topic in the on-device chip industry over the past year, and one of the biggest technical bottlenecks to running larger models on phones.
2-bit quantization compresses model weights by 6–8x so they fit in phone flash storage: 4GB of memory can hold a 16B model, and with MoE plus activation-range constraints, 32B; with 8GB, model parameters can scale to 60B.
With memory prices soaring and on-device resources always constrained, low-bit quantization has become the threshold that decides phone makers' future competitiveness. As large-model capabilities keep advancing, whoever can make phones smarter at lower cost soonest will hold the initiative in the large-model era.
The arrival of BitCPM-CANN marks another leap in the ability of ModelBest's "small but mighty" efficient models to land on devices, and, through open source, offers a new answer to China's on-device large-model race.
A 6x memory dividend that breaks the on-device AI ceiling
For a long time, the physical bottleneck of memory — capacity, bandwidth, and cost — has been the toughest challenge in bringing large models to scale, especially onto phones, PCs, cars, and other end devices.
Goldman Sachs recently raised its forecasts sharply: in the semiconductor demand chain set off by AI servers, memory prices are seen rising by as much as 280%, making memory one of the tightest resources in the global AI supply chain.
The industry's traditional answer is post-training quantization (PTQ): train the model at high precision (such as BF16), then compress its weights to INT8 or INT4.
This is essentially after-the-fact remediation, an engineering compromise that trades precision for memory; the harder the compression, the greater the performance loss.
BitCPM-CANN overturns that approach.
It takes the more demanding but more effective route of quantization-aware training (QAT): instead of being compressed passively after training, the model learns from the very start how to carry and express knowledge with 1.58-bit ternary weights (-1, 0, +1).
This is no longer simply discarding precision. It lets the model grow natively under an extremely low bit-width constraint, forcing every bit to deliver the greatest information density and knowledge-carrying efficiency.
In other words, weight precision no longer depends on bit width but on how much knowledge each bit carries. Low-bit training is therefore not just an engineering trick to save memory but a new paradigm for carrying knowledge in weights.
BitCPM-CANN was compared head to head with the full-precision MiniCPM4 family at the same sizes on 11 tasks covering common sense, reading comprehension, subject knowledge, math, and reasoning. The results for the four sizes — 0.5B, 1B, 3B, and 8B — are below:
The results show that three of the BitCPM-CANN sizes retain 95.7%–97.2% of capability; even the 0.5B model, with the lowest retention, stays above 90% (90.1%).
This shows that the low-bit training approach is systematic, scalable, and reproducible in engineering terms.
The impact of this paradigm shift on the phone and chip industries is concrete and disruptive:
For the phone industry, a 6x memory dividend means an 8B-parameter BitCPM-CANN model can run easily on today's mainstream flagship phones.
Combined with a mixture-of-experts (MoE) architecture, putting 50B or even 100B-parameter models on end devices — a fantasy in the BF16 era — now has a clear path to realization.
This raises the capability ceiling of on-device AI to a whole new level. The same applies to PCs, cars, and other terminals.
For the chip ecosystem, 2-bit is what every mainstream on-device chip vendor has been pursuing this year. Some already support 2-bit inference, and BitCPM-CANN fills the gap on the model side.
For example, the new generation of on-device chips represented by Qualcomm Snapdragon 8 Gen 4 already supports 2-bit inference natively at the hardware level. But ready hardware needs high-quality, directly deployable low-bit models to match.
BitCPM-CANN precisely fills the large supply-side gap in low-bit models for on-device chips, providing crucial ammunition for hardware-software co-development.
For AI applications, memory costs fall sharply.
With global HBM (high-bandwidth memory) prices climbing steadily on surging AI server demand, the 6x memory optimization BitCPM-CANN offers is a real cost-reduction plan that requires no extra hardware investment.
It lets enterprises raise model capability or service density substantially without adding physical memory, effectively hedging against supply chain cost pressure.
On Ascend: solving ultra-low-bit training on domestic compute
BitCPM-CANN's other major significance is that it was trained entirely on a domestic compute platform.
From the lowest-level quantization kernels and the QAT algorithm to the full parallelism strategy and training framework, BitCPM-CANN's entire training pipeline was completed natively on Huawei Ascend.
This is the first publicly disclosed result on the Ascend platform to complete 1.58-bit training end to end and run full-precision comparison evaluations, and it pushes model scale to 8B in one step.
The bar for this work is far higher than "moving a model trained on a GPU to Ascend for inference." In the past, validating low-bit training in the domestic NPU camp usually meant finishing on CUDA first and then migrating — a long pipeline with heavy losses.
BitCPM-CANN's success means the domestic NPU camp now has its own 1.58-bit low-bit training stack for the first time.
ModelBest built a complete low-bit training foundation on the MindSpeed × Megatron-LM backbone, including environment adaptation, 32K long-sequence support, parallelism strategies, and fused kernels — a full engineering system. From now on, all low-bit training work targeting Ascend can build on the same shared infrastructure.
BitCPM-CANN answers a question the industry cares about: Ascend can not only train large models, but also deliver world-class ultra-low-bit training.
The stereotype that domestic chips can only run inference can now be laid to rest.
On-device large models: the foundation determines how high you can build
As a new member of the "small but mighty" family, BitCPM-CANN is not an isolated attempt at one technical route but the payoff of years of accumulated work and early exploration in on-device large models at ModelBest.
Constrained by device memory, compute, network, power, and heat dissipation, the capability ceiling of an on-device large model depends heavily on the strength of its underlying infrastructure innovation. Since developing its own distributed training framework BMTrain in 2022, ModelBest has worked on the foundation of on-device large models for four years — almost exactly overlapping the large-model boom set off by ChatGPT.
From the training framework BMTrain to the sparse architecture InfLLM, the low-bit quantized training method BitCPM, and the inference framework CPM.cu, ModelBest has consistently worked to build systematic, efficient on-device large-model solutions.
Back in the second half of 2024, before the industry had reached a consensus on ultra-low-bit-width QAT, we chose the ≤2-bit QAT route — a judgment that put us ahead of the industry. The BitCPM series previously released on GPUs was, at launch, the world's leading ternary model at its size: BitCPM 1B comprehensively surpassed BitNet 2B while using only 1/20 of the training compute, with training efficiency far above approaches such as Microsoft's BitNet.
Why do we keep investing in the 1.58-bit / 2-bit route, which looks extreme?
Because it addresses the two most practical constraints on deploying large models at once: the physical limits of memory capacity and bandwidth, and the constant push for parameter density in on-device deployment. Neither constraint disappears as hardware advances; both grow more urgent as models get bigger.
From MiniCPM to BitCPM-CANN, ModelBest's belief in on-device AI has never changed: rather than waiting for hardware to become powerful enough for the model, make the model smart enough for the hardware.
Beyond that, ModelBest focuses on extreme optimization for underlying hardware. We have helped Huawei Ascend and Kunpeng, as well as Cambricon, Iluvatar CoreX, and other domestic chips build and optimize their software stacks, gaining a distinctive understanding of the domestic chip ecosystem. We have consistently worked toward a complete loop of "domestic NPU + domestic model + domestic training framework."
Given that background, BitCPM-CANN is not an isolated technology ported to Ascend. It moves ModelBest's QAT training methods, 2-bit model route, and training-efficiency advantages — already systematically validated on GPUs — into a complete training foundation for domestic NPUs, further answering whether this route can form end-to-end, reusable, and sustainably scalable infrastructure on Ascend.
Unlocking the performance potential of on-device large models requires joint investment from model makers and chip makers. In this important AGI track, ModelBest has never aimed merely to take part; we are committed to being a driver and builder of on-device AI technology.
Committed to open source and systematic validation
The release and open-sourcing of BitCPM-CANN completes the loop of domestic NPU, domestic model, and domestic training framework, giving the on-device AI industry a directly usable low-bit model solution.
But the real value of a technical route lies not in impressive numbers at one isolated size, but in its systematic nature, scalability, and engineering reproducibility.
To that end, ModelBest and the OpenBMB open-source community have chosen to open-source BitCPM-CANN fully, in a transparent and rigorous way.
We hope developers can experience the real performance of domestic compute in low-bit scenarios with zero barrier, and build further innovation on top.
Not a single bit should go to waste. We invite developers around the world to join this journey to the limits of AI efficiency.
The full BitCPM-CANN 0.5B / 1B / 3B / 8B model weights are now open source. Download, reproduce, and build with us:
➤ Hugging Face
🔗 https://huggingface.co/collections/openbmb/bitcpm-cann
➤ ModelScope
🔗 https://modelscope.cn/collections/OpenBMB/BitCPM-CANN