VRAM & Performance
Before deploying MiniCPM series models, you can estimate the VRAM you need based on model size, quantization method and device configuration. We've compiled VRAM requirements for common model configurations along with device recommendations, to help you quickly judge whether your current hardware can run a model reliably.
VRAM Estimation Formulas
Model Weight VRAM
VRAM (GB) ≈ number of parameters (billions) × bytes per parameter
| Precision | Bytes per parameter | Savings vs. FP16 | Notes |
|---|---|---|---|
| FP32 | 4 | -100% (doubles) | Used for training, rarely for inference |
| FP16 / BF16 | 2 | Baseline | Standard GPU inference precision |
| INT8 | 1 | 50% | Minimal quality loss |
| INT4 / AWQ / GPTQ | 0.5 | 75% | About 1-3% increase in perplexity |
| GGUF Q4_K_M | ~0.55 | ~72% | Mixed-precision quantization |
| Ternary quantization (BitCPM4) | ~0.19 | ~90% | Extreme 1.58-bit compression |
KV Cache VRAM
KV Cache (GB) = 2 × number of layers × number of KV heads × head dimension × context length × 2 bytes / 10⁹
GQA (grouped-query attention) greatly reduces KV Cache usage by reducing the number of KV heads. MiniCPM4-8B uses only 2 KV heads (vs. 32 attention heads), so its KV Cache is only 1/16 that of a full-attention model.
Total Inference VRAM
Total VRAM ≈ model weight VRAM + KV Cache + runtime overhead (~20%)
For single-user, short-context scenarios, this simplifies to: total VRAM ≈ model weights × 1.2
VRAM Quick Reference by Model
| Model | BF16 weights | INT8 | INT4 | GGUF Q4 |
|---|---|---|---|---|
| MiniCPM5-2B | 5.03 GB | 2.52 GB (theoretical) | 1.26 GB (theoretical) | 1.56 GB |
| MiniCPM5-1B | 2.16 GB | 1.08 GB (theoretical) | 0.54 GB (theoretical) | 0.5 GB |
| MiniCPM4-0.5B | 0.87 GB | 0.43 GB (theoretical) | 0.22 GB (theoretical) | 0.3 GB |
| MiniCPM4-8B | 16.37 GB | 8.19 GB (theoretical) | 4.09 GB (theoretical) | 4.4 GB |
| MiniCPM4.1-8B | 16.37 GB | 8.19 GB (theoretical) | 4.09 GB (theoretical) | 4.4 GB |
| MiniCPM-SALA (9B) | 18.95 GB | 9.48 GB (theoretical) | 4.74 GB (theoretical) | 4.9 GB |
| MiniCPM-OCR 1.0 | 2.60 GB | 1.30 GB (theoretical) | 0.65 GB (theoretical) | ~1.36 GB (self-quantized, including vision projector) |
| MiniCPM-OCR Jev | 4.43 GB | 2.21 GB (theoretical) | 1.11 GB (theoretical) | - |
| MiniCPM-V 4.7 1B | 3.11 GB | 1.55 GB (theoretical) | 0.78 GB (theoretical) | 1.78 GB |
| MiniCPM-V 4.7 35B-A3B | 70.43 GB | 35.21 GB (theoretical) | 17.61 GB (theoretical) | 22.29 GB |
| MiniCPM-V 4.6 (1.3B) | 2.60 GB | 1.30 GB (theoretical) | 0.65 GB (theoretical) | 2 GB (CPU) |
| MiniCPM-V 4.5 (8B) | 17.39 GB | 8.70 GB (theoretical) | 4.35 GB (theoretical) | - |
| MiniCPM-o 4.5 (9B) | 18.74 GB | 9.37 GB (theoretical) | 11 GB (official AWQ) | 10 GB |
| VoxCPM2 | 4.58 GB | 2.29 GB (theoretical) | 1.15 GB (theoretical) | - |
| VoxCPM1.5 | 1.60 GB | 0.80 GB (theoretical) | 0.40 GB (theoretical) | - |
| VoxCPM-0.5B | ~1.0 GB | ~0.50 GB (theoretical) | ~0.25 GB (theoretical) | - |
Notes on the figures in the table:
- Units: every value is in GB (1 GB = 10⁹ bytes).
- BF16 weights: measured total size of the safetensors weight files in the Hugging Face repository. VoxCPM-0.5B has no public weight index, so its value is estimated from 0.5B parameters and marked with "~".
- INT8 / INT4: values marked "(theoretical)" are derived from parameter count × 1 / × 0.5; unmarked values are measured official quantized weights (e.g. MiniCPM-o 4.5 AWQ).
- GGUF Q4:
Q4_K_Mfile size; for vision models the defaultmmproj-model-f16.ggufis included. - "-": no public figure exists for that metric; estimate it with the formulas above.
- None of these values include KV Cache, vision activations or runtime overhead, so they are not minimum VRAM requirements.
Weight files: MiniCPM5-2B · MiniCPM5-2B GGUF · 4.7 1B · 4.7 1B GGUF · 4.7 35B-A3B · 4.7 35B-A3B GGUF.
Device Recommendations
Recommended models and precisions based on the available VRAM / memory of common devices:
| Device | Available VRAM / memory | Recommended models |
|---|---|---|
| iPhone / Android (4-8 GB) | 2-4 GB | MiniCPM5-2B (GGUF), MiniCPM-V 4.7 1B (GGUF); start with short contexts |
| Mac M1/M2 (8 GB) | ~6 GB | MiniCPM5-1B (BF16), MiniCPM5-2B / MiniCPM-V 4.7 1B (GGUF) |
| Mac M3/M4 (16 GB) | ~12 GB | MiniCPM-o 4.5 (half-duplex), VoxCPM2, MiniCPM4-8B (INT4) |
| Mac M4 Max (64 GB+) | 40+ GB | MiniCPM-SALA, MiniCPM-V 4.7 35B-A3B (GGUF); the latter’s BF16 weights alone exceed 64 GB |
| RTX 3060 (12 GB) | 12 GB | MiniCPM5-2B / MiniCPM-V 4.7 1B (BF16, short contexts), MiniCPM4-8B (INT4), MiniCPM-o 4.5 (GGUF), VoxCPM2 |
| RTX 3090 / 4090 (24 GB) | 24 GB | MiniCPM-o 4.5 (full-duplex), MiniCPM4-8B (BF16), MiniCPM-SALA (INT4) |
| A100 (80 GB) | 80 GB | Evaluate MiniCPM-V 4.7 35B-A3B in BF16 with additional headroom for context, images and runtime overhead |
KV Cache and Long Context
How GQA affects VRAM usage for long contexts:
| Model | KV heads | 4K context | 32K context | 128K context |
|---|---|---|---|---|
| MiniCPM5-2B | 2 | 0.18 GB | 1.41 GB | 5.64 GB |
| MiniCPM5-1B | 2 | 0.09 GB | 0.75 GB | 3.0 GB |
| MiniCPM-V 4.7 1B | 2 | 0.05 GB | 0.40 GB | 1.61 GB |
| MiniCPM-V 4.7 35B-A3B | 2 | 0.08 GB | 0.67 GB | 2.68 GB |
| MiniCPM4-8B | 2 | 0.13 GB | 1.0 GB | 4.0 GB |
| MiniCPM3-4B | 40 | 2.4 GB | 19.4 GB | OOM |
New KV estimates assume one sequence and BF16 caches. MiniCPM5-2B has 42 full-attention layers. MiniCPM-V 4.7 1B / 35B-A3B have only 6 / 10 full-attention layers; their entries count those KV buffers, excluding linear-attention states, vision processing and framework allocations. Fully resident 35B-A3B weights still include all experts, not just the ~3.5B active parameters.
Thanks to its extreme GQA design (2 KV heads vs. 32 attention heads), MiniCPM4-8B needs only 4 GB of KV Cache at a 128K context, while the traditional full-attention MiniCPM3-4B already needs nearly 20 GB at 32K. MiniCPM-SALA goes even further with a hybrid architecture of 75% linear attention + 25% sparse attention, so most layers need no KV Cache — which enables million-token context on a single RTX 5090 (24 GB).
Architecture Parameter Reference
| Model | hidden_size | intermediate | Layers | Attention heads | KV heads | Head dim | Vocabulary |
|---|---|---|---|---|---|---|---|
| MiniCPM5-2B | 2048 | 6144 | 42 | 16 | 2 | 128 | 130560 |
| MiniCPM5-1B | 1536 | 4608 | 24 | 16 | 2 | 128 | 130560 |
| MiniCPM3-4B | 2560 | 6400 | 62 | 40 | 40 | 64 | 73448 |
| MiniCPM4-8B | 4096 | 16384 | 32 | 32 | 2 | 128 | 73448 |
| MiniCPM-V 4.7 1B | 1024 | 3584 | 24 | 8 | 2 | 256 | 248094 |
| MiniCPM-V 4.7 35B-A3B | 2048 | 512 (expert) / 512 (shared) | 40 | 16 | 2 | 256 | 248144 |
For 4.7, these are language-backbone parameters; full-attention head dimensions are separate from linear-attention settings. The 35B-A3B model has 256 routed experts and selects 8 per token. Configs: MiniCPM5-2B · 4.7 1B · 4.7 35B-A3B.
MiniCPM-V 4.7 On-Device Performance
| Model | Device | Officially reported memory usage | Decoding speed | Configuration |
|---|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | iPhone 17 Pro Max | ~3 GB | 13+ tokens/s | LES expert prefetching, fine-grained layer-wise quantization, DSpark speculative sampling |
These are results from the official on-device optimization, where expert weights can be loaded from storage as needed. The ~3 GB memory footprint is neither the full model’s file size nor the VRAM required to keep all weights resident on a GPU.