A new ceiling for on-device multimodal AI: 35B-parameter MoE runs smoothly in 3GB of memory on phones and tablets, MiniCPM-V 4.7 is now open source
On October 13, ModelBest and the OpenBMB community officially released and open-sourced MiniCPM-V 4.7, the latest model in the on-device multimodal series. This update focuses on pushing the ceiling of on-device capability and extending it to everyone, and ships in two versions: MiniCPM-V 4.7 35B-A3B and MiniCPM-V 4.7 1B.
MiniCPM-V 4.7 35B-A3B is the world's first open-source MoE multimodal large model that runs losslessly and smoothly on device. It leads models of the same class across 34 authoritative benchmarks and runs smoothly on iOS, Android and other systems with about 3GB of memory, taking on-device multimodal MoE efficiency to a new level.
This article includes video content. Please view it in the original WeChat post.
Advancing the performance-efficiency frontier: a unified ceiling and broad access for on-device multimodal models
On device, where memory, compute and other resources are severely constrained, the goal is to deliver as much capability as possible within a limited budget — to pursue higher "capability density". With its 35B-A3B and 1B versions, MiniCPM-V 4.7 redraws the performance-efficiency Pareto frontier for multimodal large models at both ends:
35B-A3B: In Instruct mode, it achieves a higher overall score than Qwen3.6-35B-A3B in less inference time.
1B: Building on MiniCPM-V 4.6, it improves further and now leads Qwen3.5-0.8B by a clear margin in both score and inference efficiency.
35B-A3B flagship: a new ceiling for on-device multimodal capability
Across 34 authoritative benchmarks, MiniCPM-V 4.7 35B-A3B leads the comparable Qwen3.6-35B-A3B on core multimodal tasks — general vision (MMBench 91.72, MMStar 80.67), STEM (MMMU 80.22, MathVista 88, MathVerse 86.45), OCR/charts (OCRBench 91.2, CharXiv Reasoning 75.6) and long video (LongVideoBench 68.9) — and even surpasses the larger Qwen 27B and Gemma-4 31B. It reads images, parses documents, solves problems and watches video with equal ease.
1B little powerhouse: extreme efficiency for lightweight devices
The 1B version reaches the top tier of performance at its size with very few parameters, built for broad access across mobile devices. Against the comparable Qwen3.5-0.8B and even the larger LFM2.5-VL 1.6B / 3B, it leads across general vision, OCR, chart reasoning and multi-image understanding.
Balancing a light footprint with efficient intelligence, MiniCPM-V 4.7 1B runs smoothly without strain on entry-level on-device hardware, bringing multimodal capability to more everyday devices and usage scenarios.
Technical deep dive: how does a model 4x larger than memory run losslessly and smoothly on phones and tablets?
The full weights of MiniCPM-V 4.7 35B are about 65.6 GiB — more than 4x the total memory of a mainstream 16GB device. Making it truly "run losslessly and smoothly" on device requires doing three things at once:
It fits (low memory): the memory footprint is small enough not to squeeze the system or other apps;
It runs fast (smooth): generation is fast enough that usage never stutters;
Capability intact (lossless): no trading away precision or simplifying inference to buy speed.
The prerequisite for "it fits" comes from the model architecture. MiniCPM-V 4.7 35B-A3B uses a mixture-of-experts (MoE) architecture: the model contains many submodules called "experts", and only a small fraction is invoked for each token generated. The whole model therefore does not need to sit in memory — most experts can live on a solid-state drive (SSD) and be read in when needed.
But this creates a new speed bottleneck. Which experts each layer calls can only be determined after the previous layer finishes computing, and disk reads can only start at that point. The result: for every token generated, every layer has to stop and wait for a disk read.
To address this bottleneck, MiniCPM-V 4.7 follows its established efficiency philosophy: wring the most out of every bit of compute so the model achieves "low waiting, high output".
01 Lookahead expert scheduling: read while computing for a 5x speedup, with no loss in quality
Information inside the model accumulates layer by layer, and each layer only adds a small part on top of what it already has, so the inputs to adjacent layers are highly similar. Routing, the process of selecting experts, is essentially scoring all experts and taking the top few; small changes in the input usually do not reshuffle the ranking.
Based on this, MiniCPM-V 4.7 speeds things up on two fronts at once through Lookahead Expert Scheduling (LES): "load early, compute in parallel" and "move less, read faster".
Expert prefetching and compute scheduling: prepare ahead, read while computing
LES feeds the routing input features of the current layer directly into the next layer's routing module. While the current layer computes, it predicts which experts the next layer will use, and background threads read them from flash into memory ahead of time. By the time the next layer starts computing, most of the experts it needs are already in place — the read time is "hidden" inside the compute time. This prediction method requires no additional model training, and in most scenarios it can prepare more than 80% of the needed experts in advance, sharply cutting wait time.
LES also reorders expert computation within a layer: experts actually selected by routing and already loaded compute first, while the remaining needed experts load in parallel. By overlapping compute and reads as much as possible, it further reduces load waits and GPU idle time.
The key point is that the prediction only decides "who to move early and who to compute first", not "who is ultimately used". Which experts each token actually calls is still decided by the model's original routing module based on the real input; needed experts that were not predicted are loaded and take part in computation as usual. So with LES on, the model's output is exactly the same as with it off — the speedup costs nothing in precision.
Weight storage optimization: move less, read contiguously
Beyond loading ahead, MiniCPM-V 4.7 also attacks the size and storage layout of the weights, making expert loading itself lighter and faster.
First, through fine-grained layer-wise quantization, each layer gets an appropriate weight precision, cutting the storage taken by weights and the data that must be transferred during loading, with almost no impact on model capability.
Second, the team redesigned how expert parameters are laid out in the file:
Original format: the parameters of one expert are scattered across three distant locations, so each read requires multiple jumps;
New format: each expert's parameters are stored contiguously and ordered by index, so one contiguous read can fetch one or several experts, fully using SSD read performance.
By reorganizing the expert weight layout so weights are stored contiguously, the model cuts scattered reads and lets flash read the needed data sequentially and more efficiently.
The two optimizations together reduce what expert loading demands of on-device storage capacity and bandwidth. Combined with prefetching and compute scheduling, they let the model achieve a 5x speedup while largely retaining its original capability.
With all of these optimizations, MiniCPM-V 4.7 35B-A3B reaches 17+ tokens per second on an iPad Pro M4 16GB and 13+ tokens per second on an iPhone 17 Pro Max using only about 3GB of memory, achieving lossless, smooth on-device operation for an MoE model.
In real-device tests on an iPad Pro M4 16GB, LES raised generation speed by about 5x on average.
02 End-to-end efficiency: cheaper image input, faster output
Input side: earlier visual token compression, cheaper image input
MiniCPM-V 4.7 continues the early compression inside the ViT and hybrid 4x / 16x visual token compression introduced in MiniCPM-V 4.6. The in-ViT early compression, based on LLaVA-UHD v4, compresses visual tokens inside the vision encoder, reducing computation in later encoding layers and lowering visual processing cost at the source.
On that basis, the model supports 4x / 16x visual token compression ratios, letting the same model switch between two modes as needed:
4x mode (performance first): keeps more image detail, suited to document parsing and dense text recognition;
16x mode (speed first): lower VRAM use, faster first response and lower power, suited to real-time interaction.
MiniCPM-V 4.7 further improves behavior under high compression: with 16x compression on, key scores hold essentially steady — MMBench 90.87, OCRBench 90.4, OmniDocBench 93.25. That is even higher than other models in the same class achieve at 4x compression.
Output side: DSpark speculative decoding, faster output
MiniCPM-V 4.7 also adapts DSpark speculative decoding: a lightweight draft module proposes several candidate tokens at once, and the main model verifies them in parallel in one pass. The main model can thus produce several tokens per computation, and every token that is finally output has been confirmed by the main model, so answer quality is unaffected. For on-device MoE models, fixed costs such as reading experts and scheduling compute are shared across multiple tokens, markedly lowering the cost per token.
On an iPad Pro M4 16GB, across measured tasks including code debugging and implementation, syntax correction, scientific explanation, logical reasoning and math, MiniCPM-V 4.7 35B MoE generates 1.17x faster on average under the best settings.
03 Self-Replay fine-tuning: custom capabilities without sacrificing general performance
When fine-tuning a model for a specific scenario, a common problem appears: it gets better at the new task but regresses on basic abilities such as recognition, reasoning or instruction following. We therefore introduce Self-Replay Fine-Tuning, which lets the model review what it has already mastered while learning a new skill.
Concretely, before fine-tuning, the model answers a set of questions covering general capabilities, high-quality answers are selected, and these are mixed into the training data for the new task. This "review material" comes from the model itself and is closer to its own way of answering, which helps improve performance on the target task while reducing forgetting of existing abilities. The method only changes the training data — no model architecture or loss function changes are needed.
The replay data only needs to be generated once and can be reused when the model learns different new skills or undergoes multiple training runs. Compared with training methods such as online reinforcement learning or On-Policy Distillation, which require fresh sampled data for every run, Self-Replay Fine-Tuning brings the cost of customizing new capabilities close to ordinary supervised fine-tuning (SFT), while preserving the model's general performance.
Day-0 ecosystem support, from chips to toolchains
01 Day-0 chip support
Building on its long-standing chip partnership ecosystem, MiniCPM-V 4.7 has completed Day-0 support for AMD, Apple, Arm, Houmo AI, Intel, Rockchip, NVIDIA and other vendors. Through the FlagOS unified open-source AI system software stack, it has also completed Day-0 support for a wide range of AI and on-device chips, including Cix Technology, Huawei, Hygon, Iluvatar CoreX, Kunlunxin, MetaX, Moore Threads, Xiwang Intelligent, T-Head and Qingwei Intelligent, so developers can deploy and try the new model on the corresponding hardware right away.
ModelBest has also established deep collaboration with chip and computing platform companies such as Axera, D-Robotics, Huawei Ascend, Hygon, MediaTek, Qualcomm, SiEngine and UNISOC. This broad hardware-software ecosystem covers phones, PCs, cars, robots, edge devices and servers, letting MiniCPM models reach all kinds of terminal devices faster and more efficiently.
02 Inference and fine-tuning framework support
On the software side, MiniCPM-V 4.7 supports mainstream inference frameworks including llama.cpp, Ollama, MLX, vLLM and SGLang, as well as fine-tuning frameworks such as LLaMA-Factory and SWIFT, and has open-sourced its official fine-tuning code, giving developers a complete path from local trial to serving deployment to custom fine-tuning.
Fully open source, bringing efficient intelligence to every device
ModelBest has always believed that the future of large models lies not only in parameter scale but in efficiency: carrying stronger intelligence with fewer parameters and completing more complex tasks with less compute is what lets models leave the data center and reach the device in everyone's hands.
MiniCPM-V 4.7 is another practice of that idea. At one end is the capability ceiling: as the first open-source MoE model that runs losslessly and smoothly on device, it pushes on-device multimodal to a new height. At the other end is broad access: the 1B little powerhouse lets entry-level devices carry high-quality multimodal capability too.
Going forward, we will continue along the path of efficiency, safety and open source, with the vision of "wisdom that embraces all things", so that accessible general intelligence truly reaches every household and every corner of the real world.
MiniCPM-V 4.7 model weights, inference code and fine-tuning code are all open source. Developers worldwide are welcome to download, try and build with them:
➤ GitHub
🔗 https://github.com/OpenBMB/MiniCPM-V
➤ Hugging Face
🔗 https://huggingface.co/collections/openbmb/minicpm-v-47
➤ Web Demo
🔗 https://huggingface.co/spaces/openbmb/MiniCPM-V-4.7-Demo
➤ App / local client
🔗 https://github.com/OpenBMB/MiniCPM-V-Apps
➤ Self-Replay fine-tuning technical documentation
🔗 https://openbmb.cn/blog/self-replay-fine-tuning
➤ Lookahead expert scheduling (open source)
🔗 https://github.com/tc-mb/llama.cpp-omni/blob/master/docs/les/lookahead-expert-scheduling.zh.md
➤ ModelScope
🔗 https://modelscope.cn/collections/OpenBMB/MiniCPM-V-47
➤ FlagOS model adaptation image downloads
ModelScope:
🔗 https://modelscope.cn/models/FlagRelease
Hugging Face:
🔗 https://huggingface.co/FlagRelease