MiniCPM-o 4.5 Technical Report Released: Full-Duplex Omni-Modal API Now Open, Runs in Real Time on an RTX 5070
Have you ever imagined having a human-like AI assistant on your personal computer — without an internet connection, using just a single consumer-grade GPU — that can watch, listen, speak, and even proactively remind you? It perceives changes in the environment in real time, understands your intent as it unfolds, and protects your privacy throughout.
That is what MiniCPM-o 4.5 delivers. Through technical innovation, it achieves the industry's first end-to-end full-duplex omni-modal large model with only 9B parameters, making inclusive on-device AI a reality. Since its release in February 2026, it has surpassed 250,000 downloads on Hugging Face.
➤ Model use cases
🔗 https://openbmb.github.io/minicpm-o-4_5-omni/
➤ Try it online (no registration or download required)
🔗 https://minicpmo45.modelbest.cn/
Today, ModelBest, together with the OpenBMB open-source community, Tsinghua University's THUNLP lab, and the THUMAI lab, officially releases the MiniCPM-o 4.5 technical report, disclosing for the first time ModelBest's core technology for full-duplex omni-modal interaction — the Omni-Flow streaming omni-modal framework.
➤ Technical report
🔗https://github.com/OpenBMB/MiniCPM-o/blob/main/docs/MiniCPM_o_45_technical_report.pdf
Alongside the technical report, MiniCPM-o 4.5 also introduces an online demo, an omni-modal full-duplex API, the on-device installer Comni, and a demo repository.
Online demo
The online demo is a prototype web application for MiniCPM-o 4.5, showcasing three categories of application prototypes — traditional turn-based interaction, duplex voice interaction, and duplex video interaction — and fully exposing all model configurations, including prompt and reference audio settings.
The demo can be accessed directly on mobile and desktop, with queuing, recording, saving, sharing, and replay features to improve the user experience.
➤ Try it online (recommended for mobile)
🔗 https://minicpmo45.modelbest.cn/mobile/
➤ Try it online (recommended for desktop)
🔗 https://minicpmo45.modelbest.cn/
Omni-modal full-duplex API
The newly opened MiniCPM-o 4.5 API supports omni-modal full-duplex real-time interaction. In full-duplex mode, no VAD mechanism is needed to control conversational turns, making it easier for developers to build applications on MiniCPM-o 4.5.
The API uses the https://api.modelbest.cn/minicpmo45/v1/ endpoint and is currently free to use. See the API documentation for details.
➤ MiniCPM-o 4.5 API documentation
🔗 https://api.modelbest.cn/minicpmo45/docs
Windows / macOS on-device installer Comni
MiniCPM-o 4.5 has completed model quantization and inference performance optimization based on llama.cpp. In testing, an RTX 5070 with just 12GB of VRAM runs full-duplex mode smoothly (RTF0.4), greatly lowering the barrier to personal on-device deployment.
To further lower the operational barrier to on-device deployment, the desktop app Comni integrates model download, environment setup, and demo execution, with Windows and macOS versions. Download links:
➤ Windows
Hardware requirements: GPU with 12GB+ VRAM, such as RTX 5070 / RTX 5080 / RTX 5090 / RTX 4090
Github:
🔗https://github.com/tc-mb/llama.cpp-omni/releases/latest/download/Comni-Setup-win64.exe;
ModelScope:
🔗https://modelscope.cn/models/OpenBMB/MiniCPM-o-4_5-gguf/resolve/master/app/Comni-Windows-x64.exe
➤ macOS
Hardware requirements: M1-M5 Max / M5 Pro
Github:
🔗https://github.com/tc-mb/llama.cpp-omni/releases/latest/download/Comni-macOS-arm64.dmg;
ModelScope:
🔗https://modelscope.cn/models/OpenBMB/MiniCPM-o-4_5-gguf/resolve/master/app/Comni-macOS-arm64.dmg
This article includes video content. Please view it in the original WeChat post.
(The video above shows the complete deployment and operation of MiniCPM-o 4.5 on a personal laptop, including demos of full-duplex voice conversation, real-time visual understanding, and proactive reminders.)
Open-source demo repository and Linux deployment
The full-stack code for the demo above is open source. Linux users can clone the repository and deploy the complete demo service. It is also one of the first locally deployable full-duplex omni-modal interaction demo projects.
➤ Demo GitHub repository
🔗 https://github.com/OpenBMB/MiniCPM-o-Demo
Why is full-duplex the next stop for AI interaction?
Human communication is fluid and parallel. We listen and think at the same time, and we can even interrupt each other.
But in the past, AI-human interaction was half-duplex, like a walkie-talkie: it could only process after you finished speaking, and while it spoke, it could not hear your new instructions.
This mismatch in rhythm meant most users could not get a good experience from large-model products, and some gradually lost patience because of the fragmented back-and-forth. Over time, this clearly held back the deployment of large models in multimodal scenarios.
MiniCPM-o 4.5 is the world's first to introduce "full-duplex omni-modal" capability: the model can think and respond while continuously perceiving its environment (watching video, listening to audio), turning AI from a passive tool into a true assistant that can actively help people.
Behind this is the Omni-Flow streaming omni-modal framework, co-developed by ModelBest and Tsinghua University. The technical report also discloses Omni-Flow's technical core for the first time:
Simply put, it creates a shared "timeline" that aligns all information streams — vision, audio, language — onto millisecond-level time slices. Within each tiny time slice, the model completes one "perceive–think–respond" cycle.
This mechanism gives the model continuous perception and instant response at the architectural level, and is the foundation of MiniCPM-o's full-duplex capability.
In addition, MiniCPM-o 4.5 releases and remains committed to open-sourcing a locally deployable web demo, which means for developers and users:
Absolute privacy: an always-on companion AI touches a great deal of sensitive information. Keeping data local is the best privacy protection.
Reliability without a network: it works even offline. In a tunnel or out in the field, your AI assistant will not "drop the connection."
A playground for developers: the full front-end and back-end demo code is open source. You can use it to quickly build your own full-duplex multimodal applications — whether smart cockpits, accessibility assistance, or embodied AI, MiniCPM-o 4.5 can help turn your imagination into reality.
Inside the technical report: How MiniCPM-o 4.5 is implemented: MiniCPM-o 4.5 uses an end-to-end omni-modal architecture with 9B total parameters. Core designs include:
Omni-modal end-to-end architecture: multimodal encoders and the speech decoder connect tightly to the LLM through token-by-token hidden states, achieving general visual and auditory perception and speech conversation at high compression rates.
Time-division multiplexing: parallel multimodal streams are divided into sequential information groups within periodic time slices, enabling efficient streaming.
Configurable speech modeling: supports dual system prompts of text + audio; voice-cloned role-play can be achieved with reference audio and a character prompt.
Dual-mode support: the same model supports both traditional turn-based interaction and Omni-Flow omni-modal full-duplex mode.
Real-time interaction: The Omni-Flow streaming omni-modal framework: traditional multimodal models treat interaction as a series of isolated turns, while Omni-Flow reshapes it into a continuous process.
Figure 1: Evolution of interaction paradigms; MiniCPM-o 4.5 achieves the full-duplex streaming interaction shown on the far right.
As shown, Omni-Flow precisely slices and aligns visual and audio input streams with the model's text and speech output streams in time. The model no longer passively waits for the user to finish input; instead, it continuously refreshes its "worldview" at a very high frequency (for example, once per second) and decides on its own when to step in (speak or remind).
This mechanism natively supports advanced interaction behaviors such as interruption and interjection, completely removing dependence on external aids such as VAD (voice activity detection).
End-to-end architecture: How do the 9B parameters work together?
To implement Omni-Flow, the ModelBest team designed an efficient end-to-end omni-modal architecture with 9B total parameters.
Figure 2: The end-to-end omni-modal architecture of MiniCPM-o 4.5
Its core components include:
Vision encoder (0.4B): SigLIP-ViT, responsible for "seeing."
Audio encoder (0.3B): Whisper-Medium, responsible for "hearing."
LLM base (8B): Qwen3-8B, responsible for "thinking" and understanding.
Speech token decoder (0.3B): a lightweight Llama architecture, responsible for converting the LLM's "thoughts" (text) into speech units.
Vocoder: synthesizes speech units into the final waveform.
One of the architecture's cleverest choices: the LLM base generates only text tokens, while the specialized speech synthesis task is "outsourced" to a smaller, more specialized speech decoder. This avoids making the large model handle complex acoustic tasks directly, preserving its core language and reasoning abilities. At the same time, token-level dense connections across modules keep the model's capability ceiling high.
Built for real time: the TAIL speech generation scheme
A major challenge for streaming speech is latency. To make speech sound natural, models usually need to "read ahead" a large chunk of text, but that makes the output speech lag far behind user input — fatal in full-duplex scenarios that require instant interruption. The ModelBest team therefore proposed the TAIL (Time-Aligned Interleaving) scheme, which keeps the generation of each speech chunk tightly following the latest real-time text instead of letting the text run too far ahead. A lightweight "pre-look" mechanism also solves coherence across word boundaries. Ultimately, TAIL minimizes the latency between speech output and the interaction while keeping audio smooth and pleasant. Performance: a 9B model takes on the industry's best. A small parameter count does not mean weak performance. Across evaluations on multiple dimensions, MiniCPM-o 4.5 shows it can go head-to-head with SOTA large models.
Inference efficiency: In terms of VRAM, the INT4-quantized MiniCPM-o 4.5 needs only 12GB, almost half that of the INT4 version of Qwen3-Omni, making local deployment on consumer GPUs possible. In terms of performance, MiniCPM-o 4.5 also infers faster: its INT4 version reaches a decoding speed of 212 tokens/s, over 40% faster than Qwen3, with lower response latency.
General visual capability: On multiple vision benchmarks including OpenCompass and MMBench, the 9B MiniCPM-o 4.5 performs on par with Gemini 2.5 Flash.
Omni-modal and full-duplex interaction: On benchmarks requiring joint audio-visual understanding, MiniCPM-o 4.5 comprehensively surpasses Gemini 2.5 Flash and Qwen3-Omni. On the full-duplex video understanding benchmark LiveSports-3K-CC, its win rate (54.4%) leads specialized streaming video models by a wide margin.
Speech generation: In both Chinese and English, MiniCPM-o 4.5's speech generation quality (lower character/word error rates) and emotional expressiveness outperform Qwen3-Omni and the industry-leading CosyVoice2.
True full-duplex, unlimited potential: the full-duplex omni-modal large model is not a distant concept; it will spawn a series of entirely new applications, for example:
Proactive companion: gives you real-time guidance and reminders while you cook, repair, or exercise.
Accessibility assistance: acts as "eyes" for visually impaired users, continuously observing the environment and proactively announcing key information such as a green light or a nearly full cup, helping them live safely.
Smart cockpit: continuously monitors road conditions and driver state, proactively alerts "available parking space on the left" and guides parking, providing smarter and more timely safety warnings and driver assistance.
Embodied AI: as a robot's "brain," continuously perceives the dynamic environment and decides autonomously when to interact.
What these scenarios share: the need is not one-off Q&A but an AI that blends into the flow of dynamic life as a "silent observer" and "timely reminder" — something traditional turn-based dialogue models cannot do.
MiniCPM-o 4.5 is a native full-duplex model, free of dependence on VAD. That means: it supports general sound perception (environmental noise, music, and more, not just speech); it tracks visual changes faster (native full-duplex, no need to wait for the previous sentence to finish); and the AI's speech can be redirected in real time while it talks.
Of course, MiniCPM-o 4.5 still has room to improve, such as stability over long interactions and the richness of proactive behaviors. The next frontier of multimodal intelligence lies not only in expanding model capabilities but in rethinking the interaction paradigm through which intelligence is expressed. Omni-Flow and MiniCPM-o 4.5 are ModelBest's key exploration in this direction.
Openness and collaboration will continue to drive the evolution of human-computer interaction. All developers are welcome to try the model, join the discussion, and contribute code as we explore the future of interaction together!
➤ Technical report PDF
🔗 https://github.com/OpenBMB/MiniCPM-o/blob/main/docs/MiniCPM_o_45_technical_report.pdf
➤ Try it online
🔗 https://minicpmo45.modelbest.cn/
➤ GitHub Demo (with local installer)
🔗 https://github.com/OpenBMB/MiniCPM-o-Demo
➤ Hugging Face download
🔗 https://huggingface.co/openbmb/MiniCPM-o-4_5
➤ ModelScope download
🔗https://www.modelscope.cn/models/OpenBMB/MiniCPM-o-4_5
If you run into any issues installing the demo, calling the API, or using the model, or if you would like to share and discuss, scan the QR code to join our Feishu group.