How far are we from trusting AI with our work?

2026-11-03

Today's models can already search the web, write code, call tools, and even plan and carry out a sequence of tasks on their own. Yet when it comes to handing over a genuinely complex piece of work, we still can't help watching the screen.

Last October, at the CNCC on-device large model forum, Lin Yankai, associate professor at Renmin University of China, explored a question grounded in his team's years of agent research: How can agents reliably complete long-horizon tasks?

This article is based on the talk transcript and related papers. What follows is Lin Yankai's first-person account.

About 6,200 Chinese characters, a 15-minute read

In my view, trusting an agent with a piece of work comes down to whether it can keep making the right judgments over a long execution.

Looking at any single step, today's models are already capable. But once a task gets long, errors can accumulate along the execution chain. In the past, engineers wrote rules outside the model to make these judgments. Change the task, though, and the pre-set rules quickly fail. To genuinely improve long-horizon reliability, the judgments made during execution — memory, verification, attribution, correction — need to become capabilities the model learns for itself, just like calling a tool.

This view comes from our research over the past few years. We started with WebCPM, where a human demonstrated how to search the web and the model learned by following along; then ToolLLM, which learned to read documentation and call more than 16,000 APIs on its own; then XAgent, which decomposed tasks and combined tools by itself.

Along the way, models became able to do more and more on their own. They can operate browsers, write code, and use all kinds of software. But as tasks grow longer, another problem becomes thornier than whether the model can use tools: once we hand over an entire piece of work, do we dare to let go?

In the talk I used a data-competition example. Imagine giving a competition to an agent: it reads the problem, makes a plan, and starts processing the data to prepare for model training. While checking an intermediate result, it finds the table has only 900 rows, not the 1,000 rows in the plan, so it goes back to the data and the task requirements and adjusts its processing code.

What interests me is why it stopped to check at all. Code running successfully doesn't mean the data processing meets the requirements. Whether the missing records should be filtered out needs evidence before it can be judged. If this step still depends on a human reminder, then after handing the work over we'd still have to keep watching.

Many models look impressive in demos but make people afraid to leave the screen in real use — this is why. METR ran a set of evaluations that measure how long a task a model can handle, using the time a human expert needs for the same task. At a 50% success rate, frontier models can already handle tasks equivalent to nearly ten hours of expert work. But raise the bar to 80% success, and the length of tasks they can complete reliably drops sharply.[1]

Figure 1 | The METR historical evaluation chart used in the talk: task time horizon at an 80% success requirement, measured by expert completion time. Source: METR.

Getting it right once in a while and being trustworthy enough to hand work to differ by a few judgments along the way: when it produces 900 rows, does it still remember the plan said 1,000? When it sees the number is off, will it stop and verify on its own? If the result ends up broken, does it know which step lost the data, and how to fix it next time?

Which information should be kept should be decided by the model itself

To notice that the table is missing 100 rows, the agent must still remember its original expectation at the moment of checking. In short tasks this isn't a problem, but once a task stretches out, web pages read, tool results, code errors and logs from every attempt keep pouring in. By the time verification really matters, the initial requirements are often pushed far away — or lost entirely after several rounds of summary compression.

Many systems today handle this by having people define a memory process outside the model: summarize every N steps, store certain kinds of information, overwrite old content with new content when it gets too long. This amounts to asking the designer to guess in advance which sentence will be useful later.

But pre-set rules divorced from the concrete task easily lose one thing while gaining another. For example, if every step overwrites the old summary with the latest state, the model may accurately answer "what is the user's current goal" but lose early key information and no longer answer "what was the user's original goal." This also means memory updates are not simply replacing old information with new; they require judging, according to the task's needs, which information should be updated and which is worth keeping.

Since what to remember and what to delete directly determines whether a task succeeds, we might as well let the model learn to manage memory from the feedback of task success and failure.

AtomMem follows this idea. It breaks memory management into four basic operations — create, retrieve, update, delete — and gives the model a scratchpad holding the global state plus a storage space to retrieve from on demand. How to maintain these records during a task is entirely up to the model; training doesn't require annotating sentence by sentence which step should remember which sentence — only whether the task ultimately goes well.

Under the Qwen3-8B setting, AtomMem reaches an average accuracy of 58.8% on long-document understanding and multi-turn web tasks, clearly ahead of the hand-designed MemAgent (56.7%) and A-Mem (51.4%). Even when the test-time document scale is raised from the 200 documents used in training to 800 (about 110k tokens), it still keeps the lead.[2]

What's more interesting than the total score is the habits the model develops during training. Early on, it mainly relies on retrieving existing records and rarely organizes them; as training proceeds, create, update and delete operations gradually increase. The model stops being merely a passive information lookup and begins actively managing the memory it accumulates.

Figure 2 | How the frequency of each memory operation changes during training: retrieval falls while creation, update and deletion rise. Source: talk materials.

Among them, "update" is especially critical. Remove it alone, and the model's scores on different tests drop by 4.9 to 7.2 percentage points. "Delete" matters little in these tasks because they mainly accumulate information; once the number of memory entries is capped, deletion and update become much more important. Which operation matters most depends on the task itself — exactly why memory strategy should be learned, not hard-coded.

Of course, actively adding and deleting records is one path. But in other scenarios, no matter how good the compression, some details will inevitably be lost. Take robot manipulation: where an object was just moved, which lid is covering the target, how many times the arm has already grasped and released — all of it is buried in a continuous video stream. At the moment an action happens, it's hard to know which frame will matter minutes later.

So why not simply keep the raw history and read it when it's time to act?

SimpleMemVLA first did the capacity math: under its video encoding, 60 seconds of history takes about 5.6k tokens, and a 262k context can hold roughly 45 minutes of video. Since it fits, it keeps timestamped video history directly, uses streaming prefill to reuse computation across adjacent steps, and cuts single-step decision latency from 1.02 s to 0.68 s. On the RoboMME benchmark, this approach reaches an 88.3% success rate; retrieval, compression and recurrent-state memory methods reproduced under the same backbone and training setup reach at most 31.5%.[3]

But a high success rate doesn't fully settle the question — did the model really understand the past frames, or did it just get lucky with the current frame?

To verify this, the experiments ran an intervention control: keep the current frame and everything else unchanged, and modify only the historical video. In a "which lid covers the red block" task, hiding the key evidence in history changed the model's output 86% of the time; hiding irrelevant segments left the output unchanged. In a counting task, removing one completed grasp-and-place from the historical video made the model undercount by one in all 16 trials. These modifications never appeared in training; the model's parameters were not updated, yet it adjusted its behavior according to history.

Figure 3 | Modifying only the historical video to see whether the output changes with the key evidence. After removing one completed grasp-and-place, the inferred count drops by one in all 16 trials. Source: talk materials.

Whether it's adding and deleting records on the fly like AtomMem, or keeping raw video until action time like SimpleMemVLA, the core is the same: return the decision of what to keep and what to use to the model. Only when the evidence remains intact in the model's hands can later verification be grounded in something checkable.

How to check the result is not something a prompt can solve

Even if the model remembers the original requirements, it still has to be able to check the result. This sounds like something a prompt can solve — just tell the model to "check its own work" after finishing, right?

But consider a very simple task: add speaker notes to every slide. For a human, confirming that everything is done means flipping from beginning to end; but to conclude "not done," you only need to flip to one slide, see the notes are empty, and decide on the spot.

Yet in VeriCUA's tests, a verifier driven only by a prompt demanded checking every note on every page against the original document, flipping back and forth between slides and document, and in the end still didn't give a conclusion. It had the tools to flip pages, but did not know what "enough evidence, time to stop" meant.

Figure 4 | Verifying whether every page has notes: finding one missing page is already enough to conclude the task is incomplete. Source: talk materials.

This problem doesn't disappear as models get bigger. Across desktop, mobile and web environments, when models from 4B small models to GPT-5.4 are merely prompted to check proactively, agreement rates all fall between 68% and 70%; adding a few examples to GPT-5.4 improves it by less than 1 percentage point.[4]

The model isn't incapable of inspecting the environment; it has never learned what to check first, and at what point it has checked enough.

So VeriCUA separates "deciding" from "inspecting": the module that operates the environment to gather evidence (Probe) is fixed, and training focuses on the judge that decides "where to look and when to stop" (Judge). Training considers not only whether the judgment is correct but also the cost of checking: stopping decisively after finding one missing page earns a much higher reward than flipping through ten pages before reaching the same conclusion.

After training, the 4B verifier reaches an average cross-environment agreement rate of 74.1%, 5.4 percentage points above GPT-5.4 with examples; the 1B version also beats all prompt-only verifiers; and Probe consumes about half as many tokens for evidence gathering.

More importantly, putting this verification-trained judge back into the training loop to score the working agent raises the agent's own task completion rate (Pass@4) from 43.93% to 47.14%; even on cross-application tasks unseen in training, completion rises from 17.57% to 22.97%. When the yardstick for acceptance is accurate, the model learns to do the work more efficiently too.

VeriCUA is trained only on desktop environments, yet transferred directly to mobile and web environments, it still has the highest agreement rate. Behind this is a more general pattern: evaluation ability tends to transfer more easily than task-execution ability.

A model that does the work has to learn how to operate every specific step, and may fail as soon as the software interface changes; a model that evaluates looks at "is the current state closer to or farther from the goal." As long as this judgment generalizes across tasks, there is no need to rewrite a scoring rule for every new kind of work.

The earlier AgentRM already validated this idea. We fine-tuned an execution model and an evaluation model separately on a single task: after fine-tuning the execution model, performance on other tasks generally dropped; after fine-tuning the evaluation model, most unseen tasks improved as well. Based on this, we trained an 8B evaluation model that at inference time helps the execution model choose the best of five candidate solutions. It raised the total score of an un-fine-tuned LLaMA-3-8B across nine tasks from 52.7 to 61.5, and when paired with the much larger LLaMA-3-70B it raised the latter's total by 12.6 points.[5]

In other words, evaluation doesn't have to wait until the whole job is done. The longer the task, the more it needs someone to take a look along the way: did the step just taken move closer to the goal, or has it already gone off track?

After hundreds of steps the task fails — where exactly did it go wrong

Spotting that something is wrong, however, is only the first step. Later, in the AgentCPM series, we had models write long reports, operate phone interfaces, or run hundreds of steps searching for information. Once tasks get long, the most painful training problem shows up: the model grinds through hundreds of steps, and the only feedback it gets is a bare "success" or "failure." Several attempts end with the same result, and the model learns nothing from them.

One outcome like that cannot answer which step went well and which step ruined everything.

Like the table at the beginning that was missing 100 rows. Reading web pages, downloading data, writing code, running training — the great majority of the dozens of actions were fine. If those 100 rows were lost during some table merge, marking the entire trajectory as a failure still doesn't tell the model what to change next time.

Analyzing failed trajectories, FocalRL found a pattern: most steps have little effect on success; the model's expectation of "can this still succeed" (its state value) usually drops sharply around a small number of key actions. And on failed trajectories, earlier mistakes also drag down later actions that were actually correct, causing them to be judged "bad" as well.

Figure 5 | State value during a failed execution drops around a small number of key actions. Source: talk materials.

Since the problem lies in those few steps, FocalRL concentrates its sampling budget where it matters: first locate the key error, return to the moment before it went wrong, try several alternative moves in exactly the same context, and roll forward only a limited number of steps each time. It's like discovering that data was lost at the merge step: no need to re-read the web pages from the start each time — go straight back to before the merge, remove the surrounding noise, and focus on comparing several ways of writing it.

After focusing training resources on key errors, FocalRL beats GRPO by an average of 8.4 points on five deep-research benchmarks, and improves by 5.0 and 6.4 percentage points on SWE-bench Verified and Lite respectively. At the same 4B scale, FocalRL-4B reaches 41.3% on BrowseComp, 17.2 percentage points above our earlier AgentCPM-Explore (24.1%).[6]

Of course, there is a very practical question: how are key errors found? One can't have a human go through every failed trajectory.

We asked a stronger large model to read failed trajectories and mark the erroneous steps, with access to successful trajectories and reference answers. More interestingly, when a 9B model the same size as the trained model looks for errors itself, FocalRL still beats GRPO on all five benchmarks, by an average of 4.8 points. The reason is that judging where a process went wrong is much easier than getting it right from scratch. This is the same principle as "evaluation transfers more easily than execution" from the previous section.

Admitting a mistake is useless — what matters is doing better next time

But even precisely locating the erroneous step is not the end of the story. Anyone who has used a large model has seen this: you point out that its answer is wrong, it immediately admits the mistake earnestly, rewrites the explanation longer and more polished — and then repeats the same error.

Talking about the problem doesn't mean it can actually do better next time. When asked to reflect, models tend to stare at the wrong answer they just produced; the more they look, the more they elaborate along the same dead end.

During training, CURE ties together "solve, critique, redo with a hint," and adds a crucial design: once the model has written a critique and distilled a hint for the next step, the original wrong answer is thrown away. In the redo, only the hint remains, so the model is completely free of the old answer's pull and reasons through the problem afresh.

At the same time, the quality of the critique is judged solely by whether the hint actually helps the next round get the problem right. Only if next-round accuracy truly improves does the preceding critique earn reward. In other words, what is rewarded is not "judging correctly" but "improvement" itself.

On five math tests with Qwen2.5-7B-Instruct, the CURE-trained model averages 42.0% single-round accuracy, rising to 44.9% when allowed up to eight rounds of self-check and redo; a model trained with standard GRPO, even given the same eight rounds, only moves from 41.0% to 41.4% and stalls completely after two rounds.[7]

A 2.9-percentage-point gain may not look dramatic, but the contrast shows: whether extra retries can be used depends entirely on whether the model has learned to "drop the wrong paper and find the path again with an effective hint."

Figure 6 | Multi-round self-improvement at inference after CURE vs. GRPO training. Eight rounds means up to eight rounds of self-check and redo. Source: talk materials.

In real long-horizon tasks, redoing runs into even thornier constraints. A miscalculated math problem can be erased and recomputed; a wrong piece of code can be fixed and rerun; but if the agent has already sent an erroneous email on your behalf, "throw it away and start over" is no longer an option. Which actions can still be undone, and which have already changed the external environment and can only be remedied from the current state — that too is a judgment the model must learn.

Taking down the scaffolding outside

Now let's return to the table at the beginning that was missing 100 rows.

Expected 1,000, got 900 — the mismatch is only a starting point. Perhaps those 100 rows were dirty data that should have been filtered; perhaps the merge code dropped them; or perhaps the original plan of 1,000 rows was itself wrong. Forcing the count back to 1,000 does not mean the job was done right. What the agent really needs to do is pull up the original evidence, find out where those 100 rows went, and then decide whether to change the code, change the plan, or keep the current result.

And when we hand a job to an agent in daily work, we usually give only a final goal, not a manual spelling out "verify the row count at step N, check the fields at step M." If it still needs a human beside it to point out "100 rows are missing here" or "go back to step three and rerun," then we still won't dare to leave the screen.

Of course, the studies above are still separate explorations of four links — memory, verification, attribution and correction — and cannot simply be stitched together into a fail-safe system overnight. The truly hard test ahead is to put these abilities into the same long-horizon pipeline and see whether they can cooperate reliably on unseen new tasks.

The direction these threads point to is already clear. In the past we built scaffolding around models, using human rules to decide what to remember, what to check, and how to fix things; moving from "human-designed" to "autonomously acquired" means taking that external scaffolding down, bit by bit, so the model grows these judgments itself through interaction with the environment.

When one day, faced with a different table or a new kind of work, it can stop on its own at the moment it produces 900 rows and go back to the original requirements — only then will we probably be able to hand work over with confidence.

Sources

[1] Measuring AI Ability to Complete Long Tasks. METR, 2025.

[2] AtomMem: Learnable Dynamic Agentic Memory with Atomic Memory Operation. EMNLP, 2026.

[3] SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models. arXiv, 2026.

[4] VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards. arXiv, 2026.

[5] AgentRM: Enhancing Agent Generalization with Reward Modeling. ACL, 2025.

[6] FocalRL. Microsoft Research open-source project, 2026.

[7] CURE: Critique-Driven Unified Reinforcement Learning for Test-Time Self-Improvement. ACL, 2026.