Cameras that write daily reports, robots that rap: MiniCPM Spark Program S1 wraps up
A camera that automatically collects the moments worth watching, video dubbing you can rework line by line until it sounds right, a mini control tower on your desk announcing real flights. This season, developers plugged on-device models into one concrete scenario after another.
On August 26, the OpenBMB open-source community launched the MiniCPM Spark Program S1, inviting developers in China and abroad to share their optimization practices in model invocation, device-cloud collaboration and inference deployment. S1's challenge track has now officially wrapped up.
This challenge track produced 16 winning entries, with participants from China, Spain, Italy, Australia, the United States and elsewhere. The entries range from home assistants and voice interaction devices to quantization tools, mobile model packages and training experiment logs.
From solving concrete problems close to home to publishing tools, code and experiment records, these projects also give the next wave of developers a place to start. Here are a few representative entries.
On-device models, starting with the things close to home: a home camera with an assistant that sorts the footage
A home camera records all day, but very little of it is actually worth opening. Developer Wei Dongdong built a local service on a Mac mini M4 that gives the two cameras in his home a vision assistant, automatically compiling the moments worth watching each day into GIFs and a daily digest.
Feeding every captured frame straight into a vision large model would generate a huge amount of wasted inference. To cut unnecessary inference, the author first uses lightweight object detection to filter frames, then passes the content that needs deeper understanding to MiniCPM-V. By his own records, vision model calls dropped from about 720 per day to 60–70.
Beyond chaining models together, the author also documents how to handle false triggers in conventional frame-difference detection, how to time recordings precisely, and how to merge duplicate notifications. Developers looking to cut inference costs in local security or everyday recording scenarios can work through his input filtering conditions and trigger logic point by point.
Excerpt from the home vision assistant's daily digest, from the author's public practice write-up
➤ Read the full write-up
🔗 https://mp.weixin.qq.com/s/p2AEerVqPNte6Z3OB0xRTQ
A small screen becomes a desktop tower that talks
Developer 电磁波Studio used an ESP32-S3 development board to build a mini aviation radar that sits on the desk. The small screen draws flights and headings within 120 nautical miles in real time, while the device plays control-tower speech with a radio-like texture, turning an ordinary microcontroller into an immersive desk ornament.
Demo GIF from the tower radar author's repository
The system links hardware and software as separate layers: the hardware side calls the model API, retrieves the script and synthesized speech, then plays them locally (the demo ships with simulated flight data by default).
The development board handles flight display and audio playback, calling MiniCPM5-2B through the API to generate tower scripts and then VoxCPM2 to synthesize speech. The demo uses simulated flight data by default, and the author has also published the hardware selection, firmware and build instructions as a reference for similar custom announcement devices.
➤ View the project and build instructions
🔗 https://github.com/emwstudio/esp32-radar
An AI magic mirror that sees you and keeps the conversation going
Also from 电磁波Studio, the AI magic mirror lets users face the camera and talk with a digital human on screen about whatever is in front of them. The author deployed visual understanding, speech recognition, speech synthesis and digital human rendering on a single RTX 4090 on AutoDL, recording about 21.7GB of VRAM usage, with dialogue decisions handled by the DeepSeek API. After the user speaks, the system fetches visual information as needed and has the digital human respond with voice and matching lip movement.
This article includes video content. Please view it in the original WeChat post.
Turning these capabilities into one complete interaction also means handling the calls and handoffs between models. In the project, DeepSeek handles dialogue and calls MiniCPM-V when it needs to look at the scene; VoxCPM2 synthesizes replies into speech, and the digital human component then renders expressions and lip movement from the audio.
The biggest risk in chaining multiple models is high latency and components that do not stay in sync. To make face-to-face conversation feel natural, the author also handled voice consistency across turns, audio-visual synchronization for long replies, and playback interruption when the user cuts in, so the capabilities work together more smoothly. The implementation is public in the code repository.
➤ View the implementation code
🔗 https://github.com/emwstudio/VoxEMW/tree/v1.10.0
One Reachy Mini, three ways of getting along
Can the same desktop robot switch between companionship, explanation and performance? Jimmy wrote a Conversation App for the Reachy Mini Lite, running a quantized MiniCPM-o 4.5 model instance on an NVIDIA GB10, with the app managing character settings and preset voices to support three interaction modes.
Around this single omni-modal model instance, the author designed three characters with distinctly different personalities and duties in the demo:
Companion mode (Mia): uses the model's visual understanding and everyday conversation skills, instantly recognizing the screwdriver in the user's hand and chatting naturally about what is in view;
Guide mode (Teacher Ray): answers questions using imported product materials. In the demo, when asked about battery capacity not covered in the materials, it explains that the information is not available;
Performance mode (Gee): becomes a singing, dancing RAP performer. As soon as the user says "perform the theme song," the robot recognizes the command, starts the backing track and adds body movement for a complete 25-second performance.
To make the robot's movements more controllable, Jimmy kept motion choreography at the application layer: the underlying model focuses on multimodal understanding and speech expression, while the application layer catches specific commands, plays music and schedules mechanical motion.
This project clearly shows how a single local model foundation can support multiple tasks, and it also candidly records the limits found in real testing around short command recognition and knowledge base boundaries. The project is not open-source for now; the actual interaction can be seen in the demo video the author published.
Useful, and easy for others to build on
Once a project works, some questions are still worth asking: are long-text tasks stable, how do you verify training results, and can generated content be edited easily? The following write-ups leave both the questions and the exploration process on record.
A personal admin agent that puts testing into its own workflow
Benchmark scores on public leaderboards often fail to reflect how a model performs once it is wired into real business work. Developer Paul Guglielmino put MiniCPM5 directly into his personal administrative assistant scenario, using it for everyday information organization and multi-step tool calls.
In an early cloud beta test, Paul increased context length and shifted the position of key information to see whether MiniCPM5 could find what it needed, and separately checked tool calls and task completion. Later, he shared his experience running the Hermes Agent locally on an M2 MacBook Air (16GB); by his records, MiniCPM5-2B generates at roughly 20 to 24 tokens per second.
Developers who want to build a similar local assistant can follow the author's testing approach and check model performance against their own real tasks.
➤ Read the author's evaluation and run notes
🔗 https://x.com/PaulGugAI/status/2099644400674701652
Neko-OPSD, keeping the training process and the comparison on record
Does improving an on-device model's math reasoning really require an external large model to "feed" it? In the Neko-OPSD project, developer MindsRiverPonder ran a transparent set of math reasoning training experiments around the lightweight MiniCPM5-2B.
The project uses an on-policy self-distillation (OPSD) task adaptation setup: the student and a frozen teacher initialize from the same checkpoint, the student generates answers to problems, and the teacher additionally reads the reference solution to provide a probability distribution at the positions the student generated, which is used to update the student model.
Because online sampling training is easily swayed by random factors, a single run tells you little. The author started from the same model checkpoint, compared different training approaches, and repeated the self-distillation experiment with three random seeds to observe how results fluctuate.
The author kept the full evaluation answers from five experiment groups, each answering the same 620 problems four times, for 12,400 answers in total. The training data, method description and aggregation scripts are also public. Other developers can read the model's answers under different training settings one by one, check results against the scripts, and then decide how to run their own follow-up experiments.
➤ View the experiment code and full records
🔗 https://github.com/mindsRiverPonder/NekoLLM-practice/tree/main/NekoMath-OPSD
Dubedo: after dubbing, revise line by line
In video dubbing or localization, scripts, tone and timing often still need adjusting after the dub is generated. Redoing the entire piece for every change makes revisions painful.
In the Dubedo project, developer Niklas Brandin brought the VoxCPM2 speech model into a browser-based dubbing workbench. After uploading audio or video, users can check the transcript and speakers on a visual timeline and edit lines sentence by sentence.
When a segment does not sound right, creators only need to select a single sentence and regenerate it, then export once it sounds good, avoiding the trouble of reprocessing the whole piece from scratch. The workbench also offers a way to keep the original video's background audio, bringing generation, preview, single-line rework and mixed export into one interface, so model-generated voice can keep being edited and fine-tuned until it truly fits back into the original work.
➤ Open the Dubedo workbench
🔗 https://dev.dubedo.com/
And the winners are
The judging looked at entries across four dimensions: model relevance, maturity and quality, communication value, and innovation. Different works by the same developer could each be selected.
The projects above are only a few representative entries. Below is the complete winner list for this challenge track, with no particular order within each award.
Special Award
Dubedo multilingual dubbing workbench · Niklas Brandin
ESP32 talking tower radar · 电磁波Studio
Personal admin agent evaluation · Paul Guglielmino
Blazing Spark Award
Cutting inference costs for a home vision assistant · Wei Dongdong
AI magic mirror · 电磁波Studio
Dubedo multilingual dubbing workbench · Niklas Brandin
ESP32 talking tower radar · 电磁波Studio
Aircraft attitude indicator · 电磁波Studio
Neko-OPSD self-distillation reasoning · MindsRiverPonder
Rising Spark Award
FIT-GGUF precise size quantization · Scorpio-117 (SC117)
Presidio + MiniCPM PII detection · Luis Poveda Valdivia
Personal admin agent evaluation · Paul Guglielmino
Reachy Mini: one robot, three roles · Jimmy
Spark Case Award
sharky-0.5B network traffic triage · Luis Poveda Valdivia
MiniCPM-o robot mod · Alessandro Bonomo
MiniCPM News Desk · Mark Fenner
MiniCPM5-1B / 2B LiteRT-LM conversion · Luis Poveda Valdivia
Spanish multilingual zero-shot evaluation · Luis Poveda Valdivia
Congratulations to all the winning developers, and thanks to everyone who shared their projects, code and hard-won lessons during S1.
The S1 challenge track has come to a close, but new work continues. If you are also building something interesting with MiniCPM, we would love to hear about it in the OpenBMB open-source community. Whether it is a finished project or one optimization worth recording, it could be the starting point for the next developer. See you next season!