1. Before We Begin

LaTeX is the digital language of mathematics — the bridge for bringing mathematical ideas into the digital world. The rewards are endless, but it seems to have a certain barrier to entry.

Using two models, MiniCPM5-1B + MiniCPM-V 4.6, I built a desktop app that can actually be installed and used every day.

TeXada is an on-device Agent for structured math formula input.

This Agent can:

• Turn natural language (e.g. "the sum of k squared for k from 1 to n") into compile-verified LaTeX;
• OCR handwritten formulas or screenshots into LaTeX;
• Repair broken LaTeX and render it with KaTeX so it can be copied — what you see is what you get.

The whole process runs on your own computer: Ollama loads the two MiniCPM models, so it works offline, and the FastAPI backend is bundled as a sidecar with the Tauri desktop app, keeping it small and lightweight. It's currently at v0.3.8, with DMG packages for macOS (Apple Silicon / Intel) and an installer for Windows x64, open-sourced under AGPL-3.0.

This article answers two questions: what real results can such small models actually achieve when combined, and what did I learn while turning them into an Agent.

---

2. Use Cases and Real Results

2.1 What Is TeXada

In one sentence: not a wrapper chatbot, but a math input-method tool Agent that calls deterministic tools.

I didn't use any of the complex, popular Agent frameworks; instead, it calls tools based on the characteristics and needs of TeX. The main loop is:

Plain Text

输入（自然语言 / 图片 / 残缺 LaTeX）        
↓  
MiniCPM5-1B Planner（规划下一步用哪个工具）        
↓  
TeX 工具执行（parse / compile / repair / diff / render / export）        
↓  
Observation（真实执行结果，回给 Planner）        
↓  
通过编译校验的结构化公式

The two models each have their own job, and these are the only two inference entry points:

Role	Model	Responsible for	Explicitly not responsible for
Planner / text	hf.co/openbmb/MiniCPM5-1B-GGUF:Q4_K_M (Ollama)	Planning, tool selection, NL candidates, complex completion	Directly repairing invalid LaTeX, simulating tool results
Vision / OCR	openbmb/minicpm-v4.6:latest	Proposing LaTeX candidates from images	Final verification, semantic diff, rendering

The two models together have fewer than 2.5B parameters and run on a laptop with 16GB or even 8GB of memory. They handle the uncertain natural-language parts.

Everything else — parsing, verification, repair, comparison, rendering and export — uses rule-based deterministic tools.

2.2 Three Real Product Paths

Natural Language → Structured Formula

Input: probability density function

Output, compile-verified:

Plain Text

f_X(x)\ge 0,\quad \int_{-\infty}^{\infty}f_X(x)\,dx=1

Input: 求 k 从 1 到 n 的 k 平方 (sum of k squared for k from 1 to n)

Output:

Plain Text

\sum_{k=1}^{n} k^2

Handwritten Formula OCR:

MiniCPM-V 4.6 recognizes a handwritten definite integral and produces a candidate, which is only marked as valid after the shared Agent runtime compiles and renders it. The vision model only proposes candidates and does not have its own planning loop, so failures are easy to localize: recognition problems belong to OCR, tool-selection problems belong to the Planner, and syntax problems belong to the validator.

Repairing Broken LaTeX:

Input: A Gaussian probability density missing its closing braces

Output:

Plain Text

f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2

repair_tex uses deterministic rules to complete the groups, re-verify and render.

What I'm rather proud of here is a divide-and-conquer sense of boundaries: this tool only does reproducible syntax repairs like adding a missing brace, and will not invent a missing denominator for the user — that's a semantic issue.

2.3 Measured Data

Measurements with local Ollama + the default models (on-device latency varies by device; the point here is mainly the difference between paths):

Scenario	Result
High-confidence NL hot path	About 5～25 ms, 0 tokens (uses a deterministic candidate, but still goes through the compile/render tools)
NL UI case with punctuation/synonyms	About 142 ms
Completing x+\alp	Outputs x+\alpha, 1 step / 2 tool calls / 0 tokens
OCR of a handwritten integral image	Recognized correctly, about 3.6～10.1 s end to end

Zero tokens yet a complete tool trace — that's the design I'm personally most pleased with in this architecture.

Being an Agent doesn't mean every input has to burn an inference call, but every candidate must pass through the same compile and render gates.

The fast path is an accelerator; it doesn't mean lower-quality generation.

2.4 Product Form

On the desktop it's an always-on tool: summoned with a global hotkey, shown at the top of the desktop for quick copying (no window switching while writing a paper), with browsable history and run logs, a unique run ID and full Agent trace for every request, and all data exportable for backup.

The repository includes three real screen recordings (one each for NL / OCR / repair). There's no coding-agent or chatbot-style terminal window — it's more like a minimalist natural-language-to-LaTeX input method.

---

3. Agent Lessons and Reusable Methods

Of course, I also hit some pitfalls and took some detours, and I believe some of the experience can be reused and distilled. Now that the barrier to execution has come down, I'd like to share my hard-won thinking with everyone.

3.1 A Small Model Is Not a Scaled-down Large Model — the Runtime Must Be Strict

I ran into every typical failure mode of MiniCPM5-1B in an open loop:

• Malformed XML/JSON in tool calls;
• Calling the same tool over and over;
• Ignoring an error Observation and carrying on with the original plan;
• Forgetting the operator the user originally asked for after a few turns (quietly writing \sum as \int);
• Adding another layer of backslash escaping to the formula every turn;
• Taking forever to give a final answer, or giving a natural-language explanation instead of bare LaTeX.

So you can't just wrap it with a prompt and trust the model to restrain itself.

That's where TeXada's hardcore Agent runtime constraints came from:

1. The Planner takes at most 3 steps by default (configurable from 1 to 8);
2. A normalized fingerprint is computed for every tool call, and repeated calls trip the circuit breaker immediately;
3. Two consecutive tool errors trip the circuit breaker;
4. Both inference requests and API requests have timeouts;
5. A final runtime guard acts as the safety net:

Even if the Planner fails completely, the runtime still forces compile → repair if needed → render,

so the user always gets a result that is "compilable or an explicit failure", never half a string.

The stop reason for every step is a structured enum (render_confirmed / repeated_tool_call / tool_error_limit / max_steps_or_empty_final……), which can be counted and regression-tested.

---

3.2 The Iron Rule: The LLM Decides, the Tools Execute, the Compiler Verifies

A 1B model can decide the next step, but no judgment of correctness may rely entirely on it without a gate. The model can perfectly well output a formula that looks very convincing but actually has unbalanced braces or a downgraded operator.

TeXada builds its six tools as single-responsibility units with narrow schemas:

Plain Text

parse_tex · 
compile_tex · 
repair_tex · 
semantic_diff · 
render_math · 
export

Among them, repair_tex is not a model endpoint but a set of local deterministic rules.

The Planner recognizes that a repair is needed, but the actual repair involves no model, can be unit-tested and is reproducible.

As an aside, I once thought about designing a TeX2TeX repair model for this need. Maybe I'll keep exploring that, but for now I've kept just the two models plus rules.

3.3 Binding Deeply to MiniCPM5's Native Protocol

This is the most MiniCPM-specific lesson. TeXada supports two forms of tool calling at the same time:

• OpenAI-style tool_calls returned by SGLang (--tool-call-parser minicpm5);
• MiniCPM5's native XML function calling on the Ollama/GGUF path.

Both are normalized in agent / protocol.py into the same PlannerTurn structure, so downstream code doesn't notice any difference. Another practical detail: MiniCPM5 sometimes puts the answer in the reasoning field and leaves content empty, so the model wrapper layer must be able to extract LaTeX from both fields; otherwise it gets empty output and misjudges it as a failure.

3.4 Guarding Against Syntactically Correct but Wrong Answers: The Operator Drift Guard

The most insidious failure of a small model isn't a crash but answering wrong fluently: the formula is perfectly valid, but the user wanted a sum and it gave an integral. Divide and conquer matters, but fixing semantic errors matters just as much. For this, TeXada has a Level 0 Guard:

• Presence / downgrade checks for the expected operator (if there should be a \sum, it can't turn into an \int);
• Failed anchors are fed back into a bounded Planner retry loop;
• When all Planner turns still drift, a final deterministic template recovery is allowed for specific narrow patterns.

The accompanying Level 1 is the semantic diff: it normalizes formulas into mathematical semantic units and compares before/after using role-aware weighted tree alignment. It explicitly doesn't pretend natural language is a reference AST — structural differences are only computed when two real formula versions exist. The diff outputs an edit script, a weighted cost and a [0,1] similarity, which naturally serves as a reward signal for later evaluation or training.

3.5 The Most Painful Lesson: Deterministic Tools Need Budgets Too

Before v0.3.8, I had always assumed deterministic modules were inherently reliable.

Testing proved me wrong, with three failures that ordinary user input could trigger:

• A valid 3151-character chain of nested \frac broke V8/KaTeX, the fault-tolerant parser and the recursive serializer;
• The O(m·n) semantic diff took about 65 seconds at 1200 nodes, and hit SIGBUS outright at 2000 nodes;
• Tool calls themselves had no execution limits.

Where did these inputs come from? They were generated by the model. Determinism only guarantees reproducibility, not bounded computation.

The fixes in v0.3.8 (ADR-010):

• A nesting-depth pre-check before any V8/KaTeX call (limit 100), plus separate depth budgets and a RecursionError fallback for parsing/serialization/diff;
• When the semantic diff exceeds a budget of 10,000 child-node pairings, it explicitly degrades to linear alignment and reports degraded: true, instead of hanging the request;
• All TeX tools moved to worker threads with a 10-second wall-clock timeout;
• Self-healing on V8 context failure (rebuilt up to 3 times).

Takeaway: Every deterministic tool that processes model-generated input must have explicit time, space and structural budgets; when a limit is exceeded, return an explicitly degraded result.

3.6 Observability Is the Other Half of an Agent

Every request (successful or failed) has a unique run_id and is stored in a log file that can be exported. The log records the model, latency, tokens, tool call sequence, stop reason and full trace.

For example, a successful HTTP request and a valid formula are two independent states — failed requests still go into the run log, but are never faked as a successful history.

This Request Ledger lets me compute the distribution of stop reasons from real logs and turn frequent failure cases into deterministic rules or test fixtures, forming a closed "use → observe → solidify" loop.

There's no harness that's done in one go — only experience accumulated from bad cases.

3.7 Summary of Lessons

1. Put hard boundaries on a small model's freedom: step limits, fingerprint-based circuit breaking on repeated calls, circuit breaking on consecutive errors, timeouts, and a final runtime guard as a safety net — none of them can be left out;
2. Hand correctness judgments to the compiler/validator, not to the model's confidence;
3. Support the model's native tool-calling protocol (MiniCPM5's XML format), and unify it with server-normalized tool_calls into one internal structure; remember to handle the reasoning-field fallback;
4. A zero-token fast path can save inference, but must not bypass verification;
5. Set up dedicated Guards for false successes such as semantic errors (operator presence/downgrade checks), not just guards against syntax errors;
6. Structure inputs and outputs into semantic units, so that each step's result is comparable, explainable and usable as a reward;
7. Deterministic tools need resource budgets too — model-generated input is untrusted input;
8. Log every request, and record request success and result validity separately.

---

4. Closing: There Is No Perfect AI — the Harness Is Built on the Ecosystem

Looking back on half a year of version iterations, in TeXada:

MiniCPM5-1B has a very precise role: it only needs to pick a small tool, understand the execution result, and decide whether to take one more bounded attempt — exactly the spot a 1B model can just reach on tiptoe while still delivering real value.

MiniCPM-V 4.6*, with just 1.3B parameters, turns handwritten formulas into editable structures, and is the ticket of admission for the entire OCR path.*

When I first saw ModelBest's vision, I was moved by the line about letting large models fly into every home. What TeXada wants to prove through an experimental Agent is the second half of that sentence: the models that fly into every home don't need to be big, but they do need someone to carefully design boundaries for them.

Open-source Repository and an Invitation to Build Together

The project is fully open source under AGPL-3.0-or-later. You're welcome to use it — and even more welcome to come back: https://github.com/CacinieP/TeXada-the-Math-Agent

I'd especially like to invite two kinds of issues (the repository has bilingual bug / feature templates, and the contribution guide is in .github/CONTRIBUTING.md):

• good case — inputs that TeXada handles in one pass with beautiful results. They'll be turned into regression tests and examples, protecting what already works;
• bad case — inputs that make the model drift, push repair out of bounds, trip the runtime circuit breakers or degrade a tool.

These are the real fuel. The protocol normalization in Section 3.3, the three failure paths in Section 3.5, and every item on the checklist in Section 3.7 all started out as some specific bad case. Every new bad case gets digested through the same pipeline: reproduce → locate the responsible layer (OCR / Planner / validator / Diff) → turn it into a deterministic rule, Guard anchor or regression fixture → ship it in the next release.

There is no perfect AI — only a harness system that keeps getting richer through cases. The reliability of a small-model Agent never comes built into the model; it comes from managing uncertainty, i.e. the harness: step budgets, operator anchors, semantic diff, resource limits. How fast this harness layer evolves depends on how many real inputs it has seen; and the scale of real inputs depends on how many people are willing to bring back the cases they run into.

So, thank you so much for reading this far. Issues and stars are welcome, forks are welcome, and you're even more welcome to come with your good cases and bad cases.

You're also welcome to carry the checklist from Section 3.7 into your own on-device Agent project — if it helps you avoid a pitfall, come back and tell us which one, and it will become the next item on the list.

For more details on the decisions, the changelog and my thinking in this project, head over to the docs folder on GitHub～

Thank you very much for reading～ Follows, comments and messages are welcome; your support is what motivates me to keep improving and sharing～