VRAM & Performance
Before deploying MiniCPM series models, you can estimate the VRAM you need based on model size, quantization method and device configuration. We've compiled VRAM requirements for common model configurations along with device recommendations, to help you quickly judge whether your current hardware can run a model reliably.
VRAM Estimation Formulas
Model Weight VRAM
VRAM (GB) ≈ number of parameters (billions) × bytes per parameter
| Precision | Bytes per parameter | Savings vs. FP16 | Notes |
|---|---|---|---|
| FP32 | 4 | -100% (doubles) | Used for training, rarely for inference |
| FP16 / BF16 | 2 | Baseline | Standard GPU inference precision |
| INT8 | 1 | 50% | Minimal quality loss |
| INT4 / AWQ / GPTQ | 0.5 | 75% | About 1-3% increase in perplexity |
| GGUF Q4_K_M | ~0.55 | ~72% | Mixed-precision quantization |
| Ternary quantization (BitCPM4) | ~0.19 | ~90% | Extreme 1.58-bit compression |
KV Cache VRAM
KV Cache (GB) = 2 × number of layers × number of KV heads × head dimension × context length × 2 bytes / 1024³
GQA (grouped-query attention) greatly reduces KV Cache usage by reducing the number of KV heads. MiniCPM4-8B uses only 2 KV heads (vs. 32 attention heads), so its KV Cache is only 1/16 that of a full-attention model.
Total Inference VRAM
Total VRAM ≈ model weight VRAM + KV Cache + runtime overhead (~20%)
For single-user, short-context scenarios, this simplifies to: total VRAM ≈ model weights × 1.2
VRAM Quick Reference by Model
| Model | BF16 | INT8 | INT4 | GGUF Q4 | Officially recommended VRAM |
|---|---|---|---|---|---|
| MiniCPM5-2B | 4.69 GiB (weights) | ~2.34 GiB (theoretical) | ~1.17 GiB (theoretical) | 1.45 GiB (file) | — |
| MiniCPM5-1B | 1.9 GB | 0.9 GB | 0.5 GB | 0.5 GB | — |
| MiniCPM4-0.5B | ~1.0 GB | 0.5 GB | 0.25 GB | 0.3 GB | — |
| MiniCPM4-8B | 14.9 GB | 7.5 GB | 3.7 GB | 4.1 GB | — |
| MiniCPM4.1-8B | 14.9 GB | 7.5 GB | 3.7 GB | 4.1 GB | — |
| MiniCPM-SALA (9B) | 16.8 GB | 8.4 GB | 4.2 GB | 4.6 GB | 1M context runs on an RTX 5090 |
| MiniCPM-V 4.7 1B | 2.90 GiB (weights) | — | — | 1.66 GiB (including vision projector file) | — |
| MiniCPM-V 4.7 35B-A3B | 65.59 GiB (weights) | — | — | 20.76 GiB (including vision projector file) | — |
| MiniCPM-V 4.6 (1.3B) | 4 GB | — | 3 GB | 2 GB (CPU) | Official: 4 GB (BF16) |
| MiniCPM-V 4.5 (8B) | ~16 GB | — | — | — | — |
| MiniCPM-o 4.5 (9B) | 19 GB | — | 11 GB (AWQ) | 10 GB | Full-duplex ≥ 24 GB RAM |
| VoxCPM2 | ~8 GB | — | — | — | Official: ~8 GB |
| VoxCPM1.5 | ~6 GB | — | — | — | Official: ~6 GB |
| VoxCPM-0.5B | ~5 GB | — | — | — | Official: ~5 GB |
New entries for MiniCPM5-2B and MiniCPM-V 4.7 use GiB (1 GiB = 2³⁰ bytes). BF16 values are official tensor sizes; GGUF values are Q4_K_M file sizes, including the default mmproj-model-f16.gguf for vision models. They exclude KV Cache, vision activations and runtime overhead, and are not measured VRAM requirements. MiniCPM5-2B INT8 / INT4 estimates use all 2,516,756,480 parameters, excluding quantization metadata and tensors retained at higher precision.
Weight files: MiniCPM5-2B · MiniCPM5-2B GGUF · 4.7 1B · 4.7 1B GGUF · 4.7 35B-A3B · 4.7 35B-A3B GGUF.
Device Recommendations
Recommended models and precisions based on the available VRAM / memory of common devices:
| Device | Available VRAM / memory | Recommended models |
|---|---|---|
| iPhone / Android (4-8 GB) | 2-4 GB | MiniCPM5-2B (GGUF), MiniCPM-V 4.7 1B (GGUF); start with short contexts |
| Mac M1/M2 (8 GB) | ~6 GB | MiniCPM5-1B (BF16), MiniCPM5-2B / MiniCPM-V 4.7 1B (GGUF) |
| Mac M3/M4 (16 GB) | ~12 GB | MiniCPM-o 4.5 (half-duplex), VoxCPM2, MiniCPM4-8B (INT4) |
| Mac M4 Max (64 GB+) | 40+ GB | MiniCPM-SALA, MiniCPM-V 4.7 35B-A3B (GGUF); the latter’s BF16 weights alone exceed 64 GB |
| RTX 3060 (12 GB) | 12 GB | MiniCPM5-2B / MiniCPM-V 4.7 1B (BF16, short contexts), MiniCPM4-8B (INT4), MiniCPM-o 4.5 (GGUF), VoxCPM2 |
| RTX 3090 / 4090 (24 GB) | 24 GB | MiniCPM-o 4.5 (full-duplex), MiniCPM4-8B (BF16), MiniCPM-SALA (INT4) |
| A100 (80 GB) | 80 GB | Evaluate MiniCPM-V 4.7 35B-A3B in BF16 with additional headroom for context, images and runtime overhead |
KV Cache and Long Context
How GQA affects VRAM usage for long contexts:
| Model | KV heads | 4K context | 32K context | 128K context |
|---|---|---|---|---|
| MiniCPM5-2B | 2 | 0.164 GiB | 1.313 GiB | 5.25 GiB |
| MiniCPM5-1B | 2 | 0.09 GB | 0.75 GB | 3.0 GB |
| MiniCPM-V 4.7 1B | 2 | 0.047 GiB | 0.375 GiB | 1.50 GiB |
| MiniCPM-V 4.7 35B-A3B | 2 | 0.078 GiB | 0.625 GiB | 2.50 GiB |
| MiniCPM4-8B | 2 | 0.13 GB | 1.0 GB | 4.0 GB |
| MiniCPM3-4B | 40 | 2.4 GB | 19.4 GB | OOM |
New KV estimates assume one sequence and BF16 caches. MiniCPM5-2B has 42 full-attention layers. MiniCPM-V 4.7 1B / 35B-A3B have only 6 / 10 full-attention layers; their entries count those KV buffers, excluding linear-attention states, vision processing and framework allocations. Fully resident 35B-A3B weights still include all experts, not just the ~3.5B active parameters.
Thanks to its extreme GQA design (2 KV heads vs. 32 attention heads), MiniCPM4-8B needs only 4 GB of KV Cache at a 128K context, while the traditional full-attention MiniCPM3-4B already needs nearly 20 GB at 32K. MiniCPM-SALA goes even further with a hybrid architecture of 75% linear attention + 25% sparse attention, so most layers need no KV Cache — which enables million-token context on a single RTX 5090 (24 GB).
Architecture Parameter Reference
| Model | hidden_size | intermediate | Layers | Attention heads | KV heads | Head dim | Vocabulary |
|---|---|---|---|---|---|---|---|
| MiniCPM5-2B | 2048 | 6144 | 42 | 16 | 2 | 128 | 130560 |
| MiniCPM5-1B | 1536 | 4608 | 24 | 16 | 2 | 128 | 130560 |
| MiniCPM3-4B | 2560 | 6400 | 62 | 40 | 40 | 64 | 73448 |
| MiniCPM4-8B | 4096 | 16384 | 32 | 32 | 2 | 128 | 73448 |
| MiniCPM-V 4.7 1B | 1024 | 3584 | 24 | 8 | 2 | 256 | 248094 |
| MiniCPM-V 4.7 35B-A3B | 2048 | 512 (expert) / 512 (shared) | 40 | 16 | 2 | 256 | 248144 |
For 4.7, these are language-backbone parameters; full-attention head dimensions are separate from linear-attention settings. The 35B-A3B model has 256 routed experts and selects 8 per token. Configs: MiniCPM5-2B · 4.7 1B · 4.7 35B-A3B.
MiniCPM-V 4.7 On-Device Performance
| Model | Device | Officially reported memory usage | Decoding speed | Configuration |
|---|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | iPhone 17 Pro Max | ~3 GB | 13+ tokens/s | LES expert prefetching, fine-grained layer-wise quantization, DSpark speculative sampling |
These are results from the official on-device optimization, where expert weights can be loaded from storage as needed. The ~3 GB memory footprint is neither the full model’s file size nor the VRAM required to keep all weights resident on a GPU.