Six talks, RMB 400,000 in prizes, one challenge: a full recap of the MiniCPM × Ascend Meetup in Beijing
On the afternoon of July 26, the Ascend AI Framework Meetup · Beijing was held at the Beijing AI Origin Community.
The event centered on MiniCPM and the Ascend ecosystem, but the discussion went well beyond the models themselves. From MiniCPM-o 4.5's full-duplex omni-modal interaction to distributed parallelism on Ascend supernodes; from automatic agent construction and self-evolution to inference performance optimization for speech models, and on to a companion robot that truly sees, listens, and can be interrupted—the talks traced the full path of large models from capabilities to systems to real-world applications.
The event also gave a systematic walkthrough of the MiniCPM × Ascend Inference Performance Optimization and Application Innovation Challenge, and followed the talks with two hours of in-depth discussion and hands-on practice.
Jing Chenzhe, head of the OpenBMB community, opened the event with remarks on behalf of the community. He briefly introduced OpenBMB's history and ecosystem—as an open-source community covering the full large-model stack, OpenBMB has always worked to lower the barrier to developing and applying large models and to make the technology broadly accessible and practical.
Jing also gave an overview of the MiniCPM × Ascend challenge. Initiated by ModelBest and OpenBMB together with Ascend and MindSpore, it sets two tracks around MiniCPM-o 4.5—High-Performance Inference Optimization and Innovative Applications—with a total prize pool of RMB 400,000. He encouraged developers in the room to take part and explore more of what large models can do in the Ascend ecosystem.
Starting from a real engineering problem
The event began with the MiniCPM × Ascend challenge.
Cai Tianchi of ModelBest traced the evolution of the MiniCPM-V and MiniCPM-o series: MiniCPM-V has advanced along the path of model lightweighting and on-device deployment, while MiniCPM-o focuses on the "time problem" in real multimodal interaction.
What changed in MiniCPM-o 4.5 is not just an added input modality. Omni-Flow puts vision, audio, and text on a single timeline, so the model can listen, see, and speak at the same time. While the model is responding, new sounds, new visual frames, and user interruptions can still flow in, and the current response can adjust as the environment changes.
Half-duplex interaction typically went "user finishes—model understands—model answers—user speaks again," with only input or output happening at any one moment. Full duplex means the model continuously perceives the environment and decides for itself when to keep listening and when to speak. What it changes is not response speed but the way humans and models interact.
For the High-Performance Inference Optimization track, the challenge is not a simple comparison of one isolated performance number.
Entries first have to meet two eligibility conditions: accuracy may drop by no more than 2 percentage points relative to the baseline, and the solution must integrate stably and run the full demo. On that basis, RTF per inference chunk serves as the core metric—the lower the RTF, the stronger the system's real-time processing capability.
That means contestants face a complete full-duplex inference pipeline: whether the model runs correctly, whether results are reliable, whether the demo is stable, and how long the model actually takes to respond once the user starts speaking.
The Innovative Applications track focuses on something else: how to turn MiniCPM-o 4.5's vision, speech, and text capabilities into a product that genuinely runs, demonstrates well, and can be interacted with.
Progress in the Ascend AI framework ecosystem
As large models move toward long sequences, omni-modal input, and agents, model size, sequence length, and expert count keep growing, and communication, parallel tuning, and memory management on supernode clusters grow more complex. Xu Shuai, Ascend ecosystem manager, presented the latest directions for the Ascend AI framework around development consistency, componentized innovation, and developer experience.
On development consistency, Ascend has partnered with the PyTorch Foundation to establish the Ascend for PyTorch community, accelerating alignment on APIs, features, and release cadence, with plans to release new PyTorch versions on Ascend simultaneously.
For supernodes, MindSpore continues to build Ascend-friendly components such as HyperParallel and AKG, strengthening distributed and graph-mode capabilities while supporting reuse by mainstream ecosystems such as PyTorch and LlamaFactory.
On community building, Ascend keeps improving documentation and courses while opening up community tasks, competitions, compute, and incentives, helping developers move from learning and using the stack to contributing to the community.
When large models move to supernodes, can programming stay as simple as a single card?
As model size, context length, and training workloads keep expanding, developers face more than "how do I write the model?" They also have to decide how to shard it, how to arrange devices, how to schedule communication, and how to trade off compute, storage, and communication when memory runs short.
In "Large-Model Training on Supernodes: Inside the HyperParallel Architecture," Huawei technical expert Chang Zherui introduced HyperParallel, Ascend's distributed parallel acceleration library for supernodes.
The problem HyperParallel sets out to solve is direct: let developers command large clusters and supernodes in a way that stays as close as possible to single-card programming.
It separates "what structure the model has mathematically" from "how the model executes on hardware," then injects parallelism and system optimizations into the model through declarative configuration, cutting the repetitive, complex, and error-prone engineering work in distributed development.
The system has three layers:
The first is a unified Tensor sharding representation, describing global tensors, device topology, and data distribution through DTensor, DeviceMesh, Placement, and Redistribute. The second adds model sharding strategies such as HSDP, TP, CP, EP, and pipeline parallelism. The third further reduces memory footprint through activation recomputation, Activation Swap, and parameter and optimizer Offload.
HyperParallel is also exploring agentic programming: capturing distributed development conventions, test procedures, commit practices, and common issues as Skills, Commands, and Rules. Developers make the key judgments, while agents follow validated workflows to analyze, modify, test, and commit code, so engineering experience in complex systems no longer stays only in a few people's heads.
From a fine-tuning foundation to agents that improve themselves
If LlamaFactory is about training and fine-tuning models with a lower barrier, PenguinHarness is about letting agents build agents automatically.
Zheng Yaowei, author of LlamaFactory and CTO of PrismShadow, offered a pragmatic observation: general-purpose agents are already capable, but what actually reaches production is rarely one all-powerful "universal agent." More often it is a set of custom agents built around a specific business, with controllable permissions, calculable costs, and the ability to keep iterating.
PenguinHarness frames the agent lifecycle as a closed loop:
Build → Run → Evaluate → Evolve
A user can describe a requirement in one natural-language sentence and the system builds the agent automatically. Once the agent is running, evaluators and optimizers analyze its execution results and turn the feedback into changes to prompts, Skills, and configuration.
Self-evolution does not mean an agent can modify itself at will. PenguinHarness sets boundaries through fixed execution rules, isolated evaluation criteria, limits on what can be changed, full trace logging, and snapshot and rollback mechanisms: capabilities can keep growing, but permissions, rules, and audit boundaries must not drift with them.
The talk also demonstrated an automatically built RAG configuration expert that produces citations, and multiple agents collaborating on evaluation and optimization.
For speech model inference, the bottleneck is often not only the model
For omni-modal models to enter real-time interaction, model capability is only the first step.
Speech generation typically goes through multiple stages such as Thinker, Talker, and Code2Wav. Each stage has different compute characteristics, batching behavior, and latency bottlenecks, and a blockage in any one of them directly affects how long the user waits to hear the first segment of audio.
In "Speech Model Inference Optimization and NPU Practice with vLLM-Omni," the vLLM-Omni community started from the speech model support matrix and walked through the inference paths for Omni understanding and generation, TTS speech synthesis, and music and sound-effect generation, as well as progress on supporting MiniCPM-o 4.5 on GPUs and Ascend NPUs.
For multi-stage speech models, vLLM-Omni presented a layer-by-layer optimization method: stage splitting, batching, graphs, asynchronous chunks, asynchronous output, stage replica scaling, and hot-path cleanup. Each layer addresses a specific bottleneck rather than applying a one-size-fits-all set of "magic parameters."
In the concurrency test shown during the talk, stacking several optimizations raised system throughput from 2.2 req/s to 11.7 req/s, cut time to first audio segment from about 5,884 ms to about 632 ms, and reduced RTF from 1.15 to 0.47.
For MiniCPM-o 4.5, vLLM-Omni has completed three-stage splitting, batched Code2Wav, asynchronous chunks, a chunked streaming vocoder, and Ascend NPU support, and the full-duplex runtime is now working end to end. Further work will cover operator fusion, asynchronous output, stage replicas, and hot-operator replacement.
This work maps directly to the challenge's goal: optimizing not a single operator but the entire path from the user's voice entering the model to the first segment of speech actually playing.
Emotion is not written only in words
The final talk of the day shifted the perspective from inference systems to real robot interaction.
Yan Xin, senior algorithm engineer at Xinyan Group, presented YRobot, an emotional companion robot built on MiniCPM-o 4.5.
The talk began with a very ordinary sentence:
"I'm fine."
Once ASR transcribes that sentence into text, the system gets only three words. But in real conversation, people simultaneously read speech rate, pauses, breathiness, gaze, facial expression, and body movement. The same "I'm fine" can mean things really are fine, or that the person doesn't want to talk right now, or even that they are covering up pressure that has been building for a long time.
Once a system compresses speech into text before handing it to a language model, information such as intonation, pauses, a cracking voice, and changes in gaze is lost irreversibly. A natively omni-modal model lets visual frames, audio frames, and semantics share a single timeline and take part in understanding and generation inside the same model.
Yan Xin summed up what full-duplex interaction means in one line:
"Full duplex isn't faster turns. It's no turns."
Real companionship often happens when both voices overlap. A user might sigh, shake their head, or suddenly change their mind while the robot is talking, and conventional turn-based systems cannot represent any of that.
But making a robot feel alive takes more than connecting a model.
On Reachy Mini hardware, YRobot integrates a microphone array, wide-angle camera, speaker, sound source localization, and a six-degree-of-freedom head, with engineering work for several real-world problems: the robot must not be interrupted by its own voice; when a user speaks from the side or behind, it must first determine the direction of the sound and then turn smoothly; when a user changes their mind mid-conversation, the old answer must stop immediately; and breathing, gaze, and random saccades must not jump like mechanical switches.
The live demo showed three details: when the user changes their mind mid-sentence, the robot stops at once; when called from the side or behind, it turns its head first and then answers; and even while speaking and moving, it does not mistake its own voice or motor noise for a user interruption.
The model, hardware, and code are all open source. The talk ended on a plain goal: make empathy a reproducible engineering discipline.
From listening to building
After the talks, the event moved into deeper interaction.
Attendees worked hands-on with HyperParallel, vLLM-Omni, MiniCPM-o 4.5, LlamaFactory, PenguinHarness, and other projects. Developers could talk directly with project members about model adaptation, inference optimization, distributed training, and agent building, and try the tools and demos on site.
Looking back at the whole event—from MiniCPM-o 4.5's full-duplex architecture to performance optimization on Ascend NPUs, from supernode distributed parallelism to automatic agent construction and self-evolution, and on to a robot that can see, hear, speak, and act—the discussion was really about one question:
How do we get large-model capabilities into real engineering systems and turn them into experiences users can actually perceive?
That is also the direction the MiniCPM × Ascend challenge hopes to explore with developers.
Whether you are stronger at model inference and system optimization, or want to build new applications with vision, speech, and text, there is a track for you in this challenge.
A RMB 400,000 prize pool, with no limit on imagination. We look forward to seeing more MiniCPM projects that run on Ascend NPUs, run fast, and that people actually want to use.