For a long time, the mainstream narrative in agent engineering has mostly revolved around code agents, strong cloud models, large contexts and complex tool calling.
Those directions certainly matter. Strong models can do complex reasoning, write code, break down long tasks and plan across multiple tools.
But lately I've been more interested in a different direction: if an agent doesn't run on a large cloud server, but instead runs in vertical scenarios such as smart homes, industrial equipment, enterprise intranet workflows or classified systems, how should the harness be designed?
In these scenarios, the problem changes.
We don't necessarily need an open-ended, strong-reasoning model, and we don't necessarily need long free-form conversations. Much of the time, what's really needed is a local, low-latency, controllable and replayable execution system.
On top of that, on-device hardware often has hard resource limits. Many devices may have only 2GB or 4GB of RAM. At that point, the focus of an agent harness is no longer just "how to get the model to complete more complex tasks", but becomes:
- How do you get a 1B model to reliably output structured JSON?
- How do you avoid repetition and infinite loops?
- How do you keep fields stable?
- How do you control context and output length?
- How do you avoid crashing under low memory?
- How do you prevent bad instructions from going straight to the device execution layer?
- How do you make failures replayable and evaluable?
That's why I'm building EdgeHome Harness.
What the project is
EdgeHome Harness is a Rust agent harness project for 1B on-device models.
Its product form is a local smart-home control hub, but the core goal isn't to replace the Mi Home app, Xiaomi smart speakers or Home Assistant. It's to validate one thing:
On low-resource edge devices, how do you constrain a small model that easily produces unstable output, repeats itself and falls into infinite loops into a local control pipeline that is truly executable, replayable and evaluable?
The model I'm currently using is:
openbmb/minicpm5:latest
模型包大小：688MB
On the Ollama route, the runtime budget after loading the MiniCPM5-1B q4 model is roughly between 1.0GB and 1.25GB. The first load, the first request, the KV cache, inference buffers and the page cache all cause fluctuations, and the peak may be higher.
So on a 2GB RAM device, the budget is tight:
MiniCPM5 / Ollama runtime: 1.0GB - 1.25GB
Rust Harness: 25MB - 64MB
SQLite / trace / executor: 32MB - 64MB
系统和基础服务: 350MB - 450MB
请求波动和缓存: 100MB - 150MB
必须保留 MemAvailable: 250MB - 350MB
In other words, 2GB can work, but it requires a lot of restrictions.
No desktop environment
Short context
Short output
Single concurrency
Low temperature
Limited retries
Limited trace buffer
Long-term memory not kept resident in RAM
Long-term memory injection disabled under low memory
Fall back to rule-only when necessary
Of course, for real deployment and demos I'd recommend 4GB RAM. 2GB is better suited as the extreme validation target for the low_memory profile.
Why Rust
I chose Rust for this project not to show off, but because an on-device harness is fundamentally a resource budget problem.
If the model runtime already takes 1.0GB to 1.25GB, the harness itself can't become a second memory-hungry service.
In the current design, the Rust harness targets an RSS of 25MB to 64MB, with a hard cap of 96MB. It should be a lightweight control layer, not a heavyweight web backend.
Rust's value here is mainly:
- Low runtime overhead
- Controllable memory
- Static binaries
- Strongly typed boundaries
- Long-running resident service
- Local SDK / runtime integration
It's not that it can't be done in TS or Python. But under an extreme budget like 2GB RAM, a Node.js or Python runtime typically consumes tens to hundreds of extra MB. That's nothing on a server, but on a 2GB board, that memory is exactly the safety margin for model fluctuations, KV cache, trace buffers and MemAvailable.
So the core reason this project uses Rust isn't the syntax; it's the deployment boundary.
The model only generates candidate JSON
In EdgeHome Harness, the model never controls devices directly.
MiniCPM5-1B is only responsible for generating candidate JSON. Whether something can actually be executed is decided by the Rust harness.
Ollama structured outputs make it easier for the model to output valid JSON, but that's not enough, because valid JSON doesn't mean a valid business action.
The model might output:
{
"intent": "turn_on",
"room": "bedroom",
"device": "air_conditioner"
}
But the harness still has to check:
- Does this room exist?
- Does this device exist?
- Is this action supported?
- Are the parameters out of range?
- Is confirmation required?
- Is automatic execution allowed?
- Does it trigger low-memory degradation?
- Is a fallback needed?
So the core principle of this project is:
ModelOutput != Command
Model output is always just a candidate structure. Only an ExecutionPlan validated by the harness can enter the execution layer.
The current main pipeline
EdgeHome Harness now has a complete main pipeline:
用户中文指令
-> Input Guard
-> Rule Parser
-> Runtime Memory
-> Context Compiler
-> MiniCPM5-1B / Ollama Candidate JSON
-> Output Governor
-> JSON Parser
-> Schema Validator
-> Semantic Normalizer
-> Device Registry
-> Policy Gate
-> Execution Planner
-> Executor Router
-> Trace Recorder
-> Eval / Replay
In this pipeline, the model is just a candidate generator.
The real business truth, device boundaries, memory, policy, execution and observability are all managed on the Rust side.
Runtime Memory: memory can't be left to the model
A small on-device model can't rely on long conversation history to maintain memory.
The reason is simple: the longer the context, the greater the memory pressure; the messier the information, the more easily a small model gets confused.
So EdgeHome Harness's memory system doesn't stuff the full chat history into the model. Instead, Rust manages structured short-term state, for example:
last_target
last_action
last_value
device alias
room alias
user preference
This lets the system handle real smart-home scenarios such as:
A little dimmer
Turn off the one from just now
Restore the previous settings
Switch to dehumidify mode
Long-term preferences are stored in SQLite and are not kept resident in the model context. Before each request, the harness injects only a very short summary relevant to the current input into the prompt.
This isn't about making the model "better at chatting"; it's about letting a small model still resolve the necessary references under low memory.
Output Governor: keeping a small model from going off the rails
A 1B model is different from a strong cloud model.
It can be quite valuable for short instructions, classification and slot extraction, but once its output is left unconstrained, it's prone to:
- Repetition
- Going off-topic
- Over-explaining
- Wrapping JSON in natural language
- Field drift
- Overly long output
- Infinite loops
So the harness must have an Output Governor.
It is responsible for limiting:
- Maximum output length
- Number of retries
- Repeated segments
- Timeouts
- JSON cleanup
- Fallback paths
This is an important difference between a small-model harness and a traditional large-model harness.
Many large-model harnesses assume the model is capable enough and focus on planning and tool calling. A small-model harness must assume by default that the model will make mistakes, so failure paths have to be designed up front.
Device Registry and Policy Gate: the security boundary can't be left to the model
The model shouldn't know real device tokens, and it shouldn't directly decide whether an action is executed.
EdgeHome Harness makes these decisions deterministically through the Device Registry and the Policy Gate.
The Device Registry is responsible for:
- Whether the room exists
- Whether the device exists
- How aliases are mapped
- Which capabilities the device supports
- Whether parameter ranges are valid
The Policy Gate is responsible for:
- Which actions may be executed automatically
- Which actions may only be dry-run
- Which actions require confirmation
- Which actions are rejected outright
- Whether to degrade under low memory or offline
This way, even if the small model outputs bad JSON, it won't turn directly into a real device action.
Trace / Replay / Eval: you can't tune a harness by feel
An ordinary prompt demo only cares whether the model got it right this time.
Harness engineering has to care about:
这次为什么对
为什么错
错在哪一层
能不能复现
下个版本有没有退化
低内存情况下能不能降级
So EdgeHome Harness records a Trace.
For every request, you should be able to see:
输入是什么
模型输出是什么
Harness 如何清洗
schema 是否通过
设备如何解析
策略为什么允许或拒绝
最终执行计划是什么
失败原因是什么
These traces can be used for Replay, and can also feed into Eval.
The goal of Eval isn't just to measure how smart the model is, but to measure whether the harness reliably governs the small model:
schema_valid_rate
intent_accuracy
slot_accuracy
dead_loop_rate
retry_rate
fallback_rate
trace_coverage
low_memory_degrade_count
This is also what sets this project apart from an ordinary JSON demo.
Where Evidence fits
At first I considered putting evidence gating into the online execution pipeline. But later I realized that smart-home scenarios are different from enterprise approval scenarios.
Scenarios like expense reimbursement, tickets and contract review can be slow, but can't be wrong. They suit strong evidence and strong gating.
Smart-home control is different. When the user says "turn on the air conditioner", the system should respond as quickly as possible. An ordinary light-switch action shouldn't be as slow as an approval workflow.
So the current design is:
Memory for runtime. Evidence for debugging. Eval for progress. Policy for deterministic hard constraints. LLM for candidate JSON only.
Evidence no longer gates every ordinary user action; instead it's used for failure analysis, trace replay, eval explanations and the release gate.
This trade-off matters. It gives the system engineering traceability without sacrificing online responsiveness.
Current boundaries
This project is not a complete commercial gateway right now.
Nor does it claim to replace the Mi Home app, Xiaomi smart speakers or Home Assistant.
The current default executor is MockExecutor, and Home Assistant is a demo backend. In the future it can be extended to MQTT, Matter, MIoT, miIO or other local device protocols, but the core harness is not tightly bound to any single device ecosystem.
More precisely, the current project is:
A reproducible engineering prototype of a Rust small-model harness
An on-device agent experiment in a local smart-home control scenario
A system for validating how a 1B model can enter a controllable execution pipeline
An engineering example of Memory / Policy / Trace / Eval for low-resource devices
Final thoughts
I didn't build this project to prove that a 1B model can replace large models.
What I want to prove is something else:
In vertical scenarios, a small model doesn't need to be able to do everything.
As long as the harness is designed correctly, the small model can be responsible only for candidate generation;
business truth, memory, policy, execution and observability are handed to a deterministic system.
This may be a more realistic route for on-device agents.
It's not about cramming a large cloud model into a small device, but about redesigning the model's responsibilities and the engineering boundaries so that a 1B model can deliver value under strict constraints.
That's the small-model harness I'm building now.