From Control-Tower Radar to Cat Water Counters: Putting MiniCPM on an ESP32
"Tower, Electromagnetic Wave Zero-Zero-One requesting landing."
When you hold down the button on the body and say this, what you're gripping is not a real aircraft control stick but a palm-sized ESP32 development board.
Release the button, and a tower transmission with a radio-like texture comes out of the board — an AI playing the role of tower controller. It not only understands your request but also factors in your current altitude, airspeed and pitch, reminding you in the tone of a tower call: "Caution, bank angle excessive."
Another camera-equipped board crouches beside the cat bowls, logging how much each of the four cats at home drinks. On the coffee machine, a small screen glowing with a yellow progress ring warns that today's calorie budget is nearly blown.
These curious little gadgets all come from this edition's MiniCPM × ESP32 open-source experiments by Firestarter developer "Electromagnetic Wave Studio." While most people still treat large models as an "advanced search box" or a "weekly-report-writing tool," Electromagnetic Wave Studio pulls them out of the chat windows on computers and phones and plugs them into physical little devices that can genuinely see, hear and be played with.
Fasten your seatbelt: AI is about to escape the chat box.
Starting with an ESP32, He Tried Connecting Large Models to the Devices Around Him
In hardware circles, the ESP32 is a familiar face. Cheap, with built-in Wi-Fi and Bluetooth, it is a common choice for smart-home projects and desktop toys.
But for Electromagnetic Wave Studio, long a developer of software and AI voice applications, hardware was unfamiliar territory. Until a month ago, while browsing an overseas open-source AI hardware project, he noticed the word "ESP32" for the first time. A quick search revealed that this microcontroller from a domestic chip maker costs only a few yuan.
He had encountered ModelBest's MiniCPM model family through voice cloning with VoxCPM2, and began to wonder: could he connect this little board with an on-device large model to build a device you could hold in your hand and talk to directly?
But putting "ESP32" and "large model" together raises an obvious question for many: the chip is so small and has only a few MB of memory — how can it run a model with billions of parameters?
Electromagnetic Wave Studio soon realized the answer wasn't to cram the model into the ESP32, but to split the work between the two sides. The compute-limited microcontroller handles connecting devices and collecting data; the model runs where it fits better — on a Mac or in the cloud.
So they split the work first.
The ESP32 connects to the physical world: it hooks up cameras and microphones to capture images and sound; reads sensor data, drives the display and plays audio; and interacts with people through buttons, touch and wireless communication.
Image recognition, speech understanding and response generation go to a model on the other end of the local network. For steps with real-time, privacy or cost constraints, MiniCPM runs locally on a Mac; when stronger recognition is needed, it calls models on ModelBest's official API platform.
This way, the board still runs on fixed logic, yet can give more specific feedback based on what it sees and hears. Hardware captures and presents; the model understands and generates; the two are linked over the network.
From Radar to Attitude Indicator: How a 2B Model Delivers Tower Announcements
Electromagnetic Wave Studio's first experiment, "esp32-radar," turns a round AMOLED screen into a desktop airport control-tower radar.
On screen, a scan line sweeps around with a fading afterglow; range rings, compass markings and runway heading lines are all drawn in. With Variflight data connected, the system shows flights within 120 nautical miles of the radar center, aircraft icons rotating with heading and tagged with flight number, altitude and speed. When a flight comes within 30 km, the system automatically triggers an announcement. A demo mode with simulated local flights is also included for offline use.
So how does a tiny device produce convincing tower phraseology?
If you hand flight data straight to a general-purpose model, it may flatly say, "Hello, an aircraft is approaching." The information is correct, but it doesn't sound like a tower call.
To make the announcements closer to real tower communications, Electromagnetic Wave Studio split the process into three steps:
Step one, generation. Depending on whether the aircraft is arriving, departing or overflying, MiniCPM5-2B phrases the announcement in the role of a tower controller.
Step two, pronunciation conversion. In real air-ground communications, digits follow strict pronunciations to avoid mishearing: 1 is read "yao," 2 "liang," 7 "guai," and 0 "dong." Large models excel at generating natural, varied text, but digit pronunciation is a zero-tolerance deterministic rule. So instead of letting the model improvise, the project adopted the principle of "generation to the model, rules to the program" — after MiniCPM writes the script, a piece of code performs the deterministic conversion to aviation pronunciation. The wording stays fresh and varied every time, while digit pronunciation is absolutely stable.
Step three, voice-over. Using a real tower recording as a reference, he cloned the controller's timbre with VoxCPM2 and synthesized speech, then applied slowdown, band-pass filtering and soft clipping so the voice sounds more like a call over a radio channel. The ESP32 plays it, and the TX (transmit) indicator lights up on screen in sync.
The model generates content, the program handles fixed rules, and the ESP32 presents the sound and visuals.
When the desktop radar first came to life and announced real flights passing overhead, it quickly drew aviation enthusiasts — and even many kids fell in love, through that screen, with this "device that lets you hear real airplanes."
Buoyed by that success, he quickly moved on to an advanced project, "esp32-attitude": the same screen became a simulated aircraft attitude indicator, upgrading the tower from one-way announcements to two-way real-time conversation.
This time the QMI8658 six-axis sensor on the board comes into play. Tilt the screen left or right, pitch it forward or back, and the device computes roll and pitch in real time and estimates relative heading; a demo flight model derives airspeed, altitude and vertical speed from the attitude, so the instruments on screen move with your hand.
When you hold the button and call the tower, the device not only streams your voice over Wi-Fi to the Mac (recognized by Qwen3-ASR and passed to MiniCPM5-2B) but also attaches the "live flight state" from that very frame to the model's context before sending. So the tower no longer hears an isolated "requesting landing"; it can judge against the instrument state: if the request contradicts the current flight state, MiniCPM5-2B calls it out directly; if the sensors show a steep climb, steep descent or large bank angle, the model gives the corresponding safety warning.
The model is no longer just a chatbot that answers; it can "feel" the motion in your hands and interact with you in real time.
Figuring Out Which Cat Skipped Water: 1B Screens First, 35B Confirms
The first two projects closed the loop between hearing and speaking, and gave Electromagnetic Wave Studio — who started with zero hardware background — a reusable path: even without deep hardware expertise, ordinary people can build handy little tools for concrete needs around them using open-source boards and large models.
Once technology stopped being the barrier, making things started from the most real pain points of daily life. For the third project, "esp32-eye," Electromagnetic Wave Studio turned from the virtual instruments on the desk to the four cats at home — three companions of five or six years, "Electromagnetic Wave," "Power Bank" and "Lightning," plus "Coffee Bean," who arrived less than a year ago.
Water intake is a daily metric many cat owners care about, but no one can stand by the bowl all day. With multiple cats taking turns at the same bowl, accurately recording "which cat, when, and how many times it drank" becomes a real puzzle.
Electromagnetic Wave Studio built an automatic cat water counter from two ESP32-S3 boards, a MacBook and the MiniCPM-V vision model — one board with an OV5640 camera guards the bowl and captures images and sound; another screen sits within reach on the desk, showing the live feed, avatars of the four cats and today's drink counts.
At first he planned to send every frame to MiniCPM-V 4.7 35B-A3B. Testing quickly exposed the problems: the 35B model is accurate but takes time to infer, so millisecond-level real-time response is hard; and cats moving around the room often just pass the bowl area without actually drinking. Pushing every pass through the cloud API to MiniCPM-V 4.7 35B-A3B would mean frequent network requests and inference costs, and continuously uploading home footage to the cloud raises privacy concerns.
That's where the value of pairing lightweight local screening with cloud-model verification became clear. He designed a tiered recognition pipeline:
Stage one (algorithmic filter): a traditional background-subtraction algorithm checks whether the frame changes enough. If nothing changes, no model is invoked, filtering out most empty shots.
Stage two (local 1B screening): when the frame changes, the on-device MiniCPM-V 4.7 1B running on the MacBook wakes up. It only decides "is there a cat," not the final identity.
Stage three (cloud 35B verification): only when the local 1B model repeatedly reports "cat present" and frame change crosses a threshold does the system create a candidate event, extract the relevant frames, call MiniCPM-V 4.7 35B-A3B through the cloud API to verify "is it actually drinking," and confirm identity against reference photos of each cat.
This division of labor came from real testing: three of the four cats are light-colored and look alike; the local 1B is well suited to high-frequency lightweight gating but poor at fine-grained discrimination among the four; handing final identification to the stronger 35B model strikes a balance between cost and accuracy.
What shows engineering maturity even more is how restrained the system is about uncertainty. When lighting or camera angle leaves the model's confidence in a cat's identity too low, the system keeps it as "unknown" instead of forcing a guess to make up the numbers; if the network is temporarily unavailable, the event enters a retry queue and never passes off the local 1B's preliminary result as final confirmation.
In 14 manually verified real drinking events, the system triggered a record all 14 times and correctly identified the cat, and the 35B verification stage successfully filtered out one case of "cat at the bowl but not drinking."
Rather than chasing a fake "100% recognition rate," it draws clear capability boundaries and lets the system know when not to conclude — that engineering restraint is exactly the watershed between a tech demo and a genuinely usable system.
And next? He already has his eye on the litter boxes at home, planning to apply the same tiered pipeline to solve the daily mystery of "which cat used which box."
The Calorie Ring on the Coffee Machine: AI Estimates, Humans Decide
While debugging the cat project, MiniCPM-V 4.7's vision performance gave Electromagnetic Wave Studio plenty of inspiration: if the model can tell apart cats with similar coats, can it identify dishes on an everyday dinner table and estimate their calories?
So came the fourth project, "esp32-intake." Unlike the earlier projects that handled complex audio and video streams, this time he took a lighter approach: snap a photo with the phone, let the model identify it, and let a small desktop screen keep reminding you.
Before each meal, you photograph the food with an iOS app; the system calls MiniCPM-V 4.7 35B-A3B to identify the dish and estimate calories. The result isn't logged immediately — the app first shows "how this meal would affect today's goal," leaving the final call to the human: you can tap "Add to log," "Re-measure," or "Not eating."
"Often people take a photo just to get an estimate first," Electromagnetic Wave Studio explains. "Hardware and AI provide a reference for the decision, but whether to eat is still up to the person."
After confirmation, the app stores the day's intake locally, computes the remaining allowance, and pushes the full state over LAN HTTP to the ESP32 screen on the coffee machine. The center of the screen shows today's remaining calories, while the outer progress ring fills clockwise: green when intake is below 70% of the target, yellow from 70% to 99%, and a striking red once the target is reached or exceeded.
The ESP32 doesn't sum data itself, nor does it understand food. It only receives the state pushed by the app and displays it steadily. The project draws responsibilities very clearly: AI estimates, humans decide, hardware keeps reminding. From an idea to a product you can use in real life over the long term, these boundaries matter just as much.
After AI Steps Out of the Chat Box
Over the past two years we have seen plenty of dazzling large-model chat windows. You type a line on the keyboard, and hundreds of florid words pour out from the other side of the screen.
But life isn't lived inside a chat window.
Looking back on four attempts in one month, Electromagnetic Wave Studio thinks the underlying logic has stayed the same: "Technology ultimately has to serve people's real needs — whether helping with everyday decisions or providing emotional value. AI does the thinking and generating behind the scenes, while hardware presents it all right at your fingertips."
Once MiniCPM is connected to an ESP32, the large model becomes, like water, electricity and Wi-Fi, a foundational capability that can be plugged into all kinds of hardware at any time. It doesn't need to cram huge parameters into every tiny chip; instead, through sensible division of labor, each part does what it does best: the ESP32 connects to physical reality, locally run models handle high-frequency lightweight perception and screening, cloud-API models handle more complex understanding and verification, and speech models give the hardware a vivid, characterful voice.
So in Electromagnetic Wave Studio's hands, AI becomes a tower announcement that sounds automatically when a plane enters the airspace, a spoken warning that incorporates gyroscope state, a cat-water card updating in real time on the desk, and the calorie ring quietly turning yellow and red beside the coffee machine.
He isn't stopping. He now plans to put an ESP32 camera and MiniCPM-V 4.7 into a desktop robot — a "workout buddy" that cheers you on in various Chinese dialects while you do push-ups. Swap in different sensors, displays and personas, and the same architecture can grow countless little devices that don't even have names yet.
The model was never the destination, only the starting point. Once model capabilities are wired into hardware you can touch, watch and keep within reach, the next surprise may be in your hands.