Self-Improvement Is Not Adaptation: How PACE-Bench Evaluates Self-Evolving Agents

2026-08-24

As works such as Claude plays robotics and Waddle bring general agents into robots and the physical world, agents have begun completing real tasks through execution, feedback, and code modification. But the real world does not stay fixed: if the environment itself changes and previously successful solutions stop working, can an agent still recover autonomously through new interactions? This is exactly the question a Self-Evolving Agent must answer as it moves into the real physical world.

To this end, the team proposes PACE-Bench (Physics Adaptation via Code Evolution), using code-driven physical design as a controllable test scenario for self-evolution. For example, a car that can drive on a road starts to skid once the surface turns to ice. The goal and interface are unchanged, but the conditions the old design relied on are obsolete. The agent must identify the invalid assumptions from new feedback and revise the design code that defines the mechanical structure and control logic until the solution works again.

The researchers compared 10 self-evolving methods across four paradigms on the full benchmark, and further evaluated larger open-source and frontier models on a selected subset. The results are not encouraging: on the full benchmark, the best performer, Reflexion + Qwen3-14B, reaches only 35.9% Pass@2. GPT-5.5 reaches 66.7% on statics tasks but still fails one-third of them. Different methods also introduce extra model calls, search, or parameter updates, but longer compute time does not mean higher scores within the same time.

Further analysis shows the bottleneck is not just whether the model is strong enough. Reflexion analyzes the failure from the feedback returned by the previous Box2D execution and re-enters the environment after a single revision. Self-Refine, although it also receives environment feedback at the outer layer, makes up to five consecutive revisions based only on model self-critique before the next validation, and its results degrade significantly. When methods add memory or search, agents tend to oscillate between clinging to the old design and exploring without direction. Even disclosing the exact parameters of the environment change did not improve the best result. But an agent that can truly self-evolve must not only know what happened in the world (Know What) but also decide accordingly whether to change parameters, structure, or control, and write code that runs in the new environment (Know How).

➤ Project page

🔗 https://thunlp.github.io/PACE-Bench/

➤ Paper

🔗 https://arxiv.org/pdf/2608.14441

➤ Code & Benchmark

🔗 https://github.com/thunlp/PACE-Bench

What really happens when a Self-Evolving Agent encounters an environment change?

Self-evolving agents update their context, memory, tools, or model parameters from interactions so that later behavior gradually improves. But existing evaluations usually keep execution conditions fixed, mainly observing whether an agent can accumulate experience within a task stream or transfer learned capabilities to other tasks. The real environment does not stay fixed. When terrain, materials, or dynamics change, a previously effective solution may fail immediately, and the agent must determine which assumptions are outdated and whether to keep revising the original solution or switch design directions.

PACE-Bench uses code-driven physical design to test this capability. Here, Python code is both an executable design and the interface through which the agent operates the simulated environment. The code itself defines the mechanical structure, part connections, and control logic. Running the code generates the corresponding physical system in Box2D, and the agent can observe from motion trajectories, task scores, and diagnostic information whether the design works. Each revised version of the code directly represents a new structural and control scheme.

After the environment changes, the task goal and code interface stay the same, but the physical conditions the old design relied on may be gone. For example, a car that can drive on a high-friction road may only spin its wheels on ice. At this point, the agent must not only determine that friction has changed but also turn that judgment into a new executable design. The whole adaptation process is:

observe failure → infer the environment change → revise the executable design → re-execute → continue iterating on new feedback

Figure 1: The Source→Target adaptation process in the S-01 bridge task. The Source design succeeds in the original environment but fails in the Target environment after Mutation; the agent then adapts through feedback, physical reasoning, and code redesign.

How PACE-Bench actually tests: from Source to Target environments

Building on this setup, PACE-Bench constructs 36 base tasks covering six physics domains:

Statics & Equilibrium

Kinematics & Linkages

Dynamics & Energy

Granular & Fluid Interaction

Cybernetics & Control

Exotic Physics

Each task first provides a feasible design in a Source environment, then systematically changes terrain, friction, material strength, or dynamics through Mutation to obtain four Target environments. This yields 180 environments and 144 Source→Target environment pairs. Before and after mutation, the task goal and code interface remain unchanged. The agent still uses the same set of Primitive APIs — the basic physical building-block interfaces provided by the benchmark — to create objects, connect structures, and apply control, and cannot directly modify the environment or evaluator.

The agent receives design code that scores 100 in the Source environment, but this code already fails in the corresponding Target environment. It has at most 20 chances to submit revised designs. Each submission is actually executed in a Box2D sandbox, and the system returns whether constraints are satisfied, task progress, and failure diagnostics, based on which the agent continues to revise. Evaluation focuses on whether it can, through this interaction loop, transform the failed Source design into a new design that adapts to the Target environment.

Figure 2: The left panel shows the six physics domains and their 36 tasks; the right panel shows the Box2D simulation of reference solutions for the 36 Source environments.

To make each environment pair a valid self-evolution test, the researchers set three conditions.

First, ensure the task is non-trivial. The Source design must score 100 in the Source environment and fail in all four Target environments, ruling out the case where copying the old code passes directly.

Second, ensure the task is solvable. Each Target environment comes with an independent reference solution validated by Box2D execution and scoring 100, so failure cannot be attributed to an unsolvable problem.

Third, by default use the Change-Hidden setting. The agent knows only a set of variable names that "may have changed," but not what actually changed, in which direction, or by how much. Constraints and visible geometric quantities are still given accurately, but hidden parameters such as friction, damping, and gravity must be inferred from execution feedback.

Thus, PACE-Bench measures two distinct capabilities at once:

Physical Inference (Know What): judging from failure phenomena and diagnostic feedback what changed in the environment.

Mechanism Redesign (Know How): turning that judgment into structures and control mechanisms that work under the target physical conditions.

How are these tasks constructed, and how is "model-generated, model-graded" avoided?

PACE-Bench separates task generation, difficulty calibration, and final scoring.

Figure 3: Task and mutation environment generation, modular implementation and reference solution derivation, and automated auditing plus human verification form a three-stage construction pipeline.

First, researchers construct 36 tasks and their four Target environments each, using modular implementations so that Mutation acts simultaneously on the environment, evaluator, and prompt. They then derive an independent reference solution for each environment and verify through Box2D that it reaches 100.

Next, researchers calibrate the tasks with automated checks, Qwen3-4B stress testing, and three rounds of human review. Tasks must neither be passable by the old design nor lack an executable solution. Feedback only indicates where, when, and by how much the failure occurred, without giving the fix directly. Claude Code participates in task construction and reference solution derivation, but final success is always determined by Box2D and the hard-coded evaluator.

How do researchers evaluate Self-Evolving methods?

The team used Qwen3-4B, Qwen3-8B, and Qwen3-14B as backbones and systematically compared 10 self-evolving methods with a Vanilla baseline, covering four mainstream paradigms:

Context-based: Reflexion, Self-Refine

Memory-augmented: ACE, ExpeL, ReasoningBank

Inference-time Search: Tree-of-Thoughts (ToT), CodeEvolve

Parameter-based: SEAL, RAGEN, TTT-Discover

Here Vanilla uses an iterative loop: generate candidate code → Box2D validation → add structured diagnostics to the revision prompt → revise again. Each round keeps the latest attempt and the best historical attempt, but uses no extra memory, extra agents, or weight updates. The other ten methods all add their own self-evolving mechanisms on top of this common loop.

Each Source→Target environment pair is run twice independently. The team mainly reports:

Pass@2: the proportion of environment pairs with at least one success in two independent runs.

Score@2: the average of each run's best score, where 100 means a full pass and a negative score means hard constraints are still unmet.

Is the benchmark saturated? Bigger models don't solve it either

The answer is clear: far from saturated. On the full benchmark, the best combination, Reflexion + Qwen3-14B, reaches only 35.9% Pass@2, with a Score@2 of 28.0. The main results also show clear scaling. For example, Reflexion's Pass@2 goes from 19.5% to 26.6% to 35.9% at 4B, 8B, and 14B. But scale growth did not bring the task close to solved.

Figure 4: Pass@2 and Score@2 on the full PACE-Bench. Parentheses show the change relative to Vanilla at the same scale.

To further test whether stronger models can close the gap, the team fixed Vanilla on 24 statics environment pairs and added Qwen3-32B, DeepSeek-V4-Pro, and GPT-5.5.

Figure 5: Vanilla results for larger models on statics tasks. Qwen3-32B's Pass@2 matches 14B. GPT-5.5 rises to 66.7%, but one-third of the environment pairs still fail to recover.

Qwen3-32B in the same series does not surpass 14B, showing that parameter scale does not reliably translate into adaptation capability. GPT-5.5 improves markedly but still does not saturate the task.

The From-Scratch control further validates task difficulty. When no Source design is given and Vanilla constructs solutions from scratch, Qwen3-4B, 8B, and 14B achieve only 11.3%–18.3% Pass@2. By environment, Source environments reach 32.3%, while the four Target environments reach only 6.4%–13.4%. Mutation indeed pushes Target environments into a harder design region that cannot be easily solved by direct generation.

The first counterintuitive result: the most complex "self-evolution" is not necessarily more effective

In the main results, the significant difference between Reflexion and Self-Refine deserves particular attention.

After each round of sandbox feedback, Reflexion first adds a dedicated failure-analysis step before revising the code. Self-Refine, before re-entering the sandbox, performs up to five consecutive internal self-revisions. Intuitively, more rounds should be more thorough. But the result is the opposite. Self-Refine significantly trails Vanilla on Qwen3-4B, 8B, and 14B, dropping from 32.0% to 7.1% at 14B while significantly increasing wall-clock time.

From an efficiency perspective, the answer differs again. Researchers compared methods on kinematics tasks using Score/Hr (average score divided by wall-clock hours):

Figure 6: Score, wall-clock time, and Score/Hr on kinematics tasks. ToT achieves the highest per-hour efficiency at 4B, 8B, and 14B.

ToT's Score/Hr reaches 45.7, 46.4, and 57.8 at 4B, 8B, and 14B, but the highest absolute scores are achieved by different methods. Self-Refine reaches only 2.0–3.6 Score/Hr, and CodeEvolve only 3.7–7.5. "Who ultimately does best" and "who improves fastest in the same time" are two different questions. Extra compute is only valuable if it turns into effective exploration.

Takeaway 1: Reflection with simulator validation is more reliable than unvalidated consecutive revision. ToT's highest per-hour efficiency also shows that higher success rates and higher efficiency are two different goals.

The second key result: why do agents lose balance between "lock-in" and "roaming"?

Why do agents still fail often despite having an initial design and continuous execution feedback?

Researchers classified failure trajectories into nine types and further analyzed the Jaccard similarity between code generated in each round. The results show two opposite trajectories. Conservative methods tend toward Design Fixation, repeatedly making local patches around an early mechanism. Aggressive methods are more likely to fall into Stagnation or undirected Exploration and fail to converge.

Figure 7: Share of error types within failed runs. The left panel aggregates by the six physics task categories, the right panel by method. Bar height indicates the composition of failure types, not the absolute failure rate across all tasks.

In Vanilla's failures, Design Fixation accounts for 41.0%. Although ACE reduces Stagnation from 12.6% to 7.5%, it pushes Fixation up to 47.1%. ReasoningBank's code also becomes increasingly similar across iterations. Retrieved experience does not automatically become transferable knowledge; instead, it may limit a stronger model's ability to re-reason from current feedback.

Search methods tend toward the other extreme. ToT's Fixation is only 0.3%, but 50.2% of its failures are Stagnation: it escapes the old design but never converges exploration into a feasible solution. CodeEvolve's crossover and mutation stay too close to the parent, with Fixation as high as 66.3%.

Takeaway 2: Self-evolving agents easily lose balance between sticking to an old design and exploring without direction. Memory-augmented methods may anchor search to early solutions, while inference-time search may escape old solutions but fail to converge reliably.

The key is not whether a module is more complex, but whether it can use new evidence to overturn wrong assumptions while retaining a stable optimization direction.

The third key result: why doesn't more information always help?

PACE-Bench hides specific physical changes by default. So could model failure simply mean it failed to guess changes in friction, gravity, or damping?

To test this, researchers set up a Change-Exposed (CE) control: telling the agent directly which variables changed and their new exact values. This removes Physical Inference (Know What) almost entirely from the task, leaving mainly Mechanism Redesign (Know How).

Figure 8: Change-Hidden (CH) and Change-Exposed (CE) Pass@2 on statics and kinematics tasks. CE helps all 4B configurations but does not raise the overall best result.

After disclosing exact parameters, the three 4B configurations improved by 2.6–8.4 percentage points. At 8B and 14B, most changes were small, and Vanilla-14B actually dropped 7.2 points. The best CH result is 17.9%, still above the best CE result of 14.6%. Disclosing parameters can lower the difficulty of Physical Inference, but does not solve Mechanism Redesign.

A model may correctly understand that "friction decreased" without knowing whether to adjust wheel structure, mass distribution, or drive mode, or abandon the original vehicle mechanism. Mechanism Redesign requires combining failure phenomena with physical constraints to judge why the original design no longer holds, then deciding whether to adjust parameters, modify structure and control, or propose a new mechanism, and implementing that judgment as executable code. This task tests physical reasoning, design decisions, and iterative validation at once; simply searching over large numbers of parameter combinations cannot capture its difficulty.

Video feedback also reveals differences between methods. At 14B, adding VLM-generated execution descriptions dropped Vanilla from 40.9% to 18.2% and Reflexion from 40.9% to 27.3%. ACE, however, rose from 13.6% to 22.7%, and ExpeL from 13.6% to 36.4%. Changes in error types suggest one possible explanation: video information disrupts the concentrated reasoning of context-based methods, but can use new evidence to correct stale experience in memory-based methods.

Therefore, methods need to judge which evidence is sufficient to overturn old assumptions and which information only brings undirected change.

Takeaway 3: More information is not always more helpful. Disclosing exact changes did not raise the best result, and video feedback has opposite effects depending on method paradigm. Knowing "what changed" (Know What) still does not equal knowing "how to redesign" (Know How).

Do different task categories share the same difficulty?

No. Aggregating all methods, Exotic Physics achieves Pass@2 of 22.2%, 25.0%, and 38.5% at 4B, 8B, and 14B, consistently the easiest to succeed at. Dynamics is the hardest overall, at only 8.2%, 9.7%, and 14.2%.

Figure 9: Model × Category Pass@2 aggregated over all methods. S/K/D/F/C/E denote statics, kinematics, dynamics, granular/fluid, control, and exotic physics, respectively.

Different categories also benefit unevenly from scaling. From 4B to 14B, Exotic Physics improves 16.3 points and Granular/Fluid improves 12.2 points, while Kinematics improves only 1.9 points. Looking at specific tasks, kinematics often requires searching combinatorial structures such as linkage topologies. Such difficulties do not disappear naturally as model reasoning improves.

Thus, category structure explains task difficulty better than model scale alone: scaling improves reasoning, but does not necessarily solve combinatorial design search in adaptation.

What does PACE-Bench teach us?

Taken together, the researchers draw three lessons for next-generation self-evolving agents.

First, self-evolution must be anchored in verifiable environment feedback. Internal reflection is valuable only when it absorbs new evidence. Real execution, precise diagnostics, and failure-specific reflection are more reliable than unvalidated multi-round self-revision.

Second, memory and search must jointly solve "break out and come back." Memory-augmented methods should not just repeatedly retrieve early high-scoring solutions; they should also record which assumptions have been refuted by physical evidence. Inference-time search should not just expand the number of branches, but identify mechanism-level differences and concentrate resources on design families with real promise.

Third, more information becomes valid evidence only when correctly absorbed. Disclosing exact physical parameters did not raise the best hidden-setting result, and video feedback helps memory-based methods while interfering with context-based ones. Evaluation should not only ask how much information the model receives, but whether it can turn evidence into executable design.

Conclusion

PACE-Bench places self-evolution in a stricter and more realistic setting: goals and interfaces stay the same, but environment changes invalidate past success, and the agent must recover autonomously from execution feedback. Across 36 tasks and 144 environment pairs, the best Pass@2 is still only 35.9%, showing this capability is far from solved. Experiments further show that reflection with environment validation beats unvalidated repeated revision, memory and search must both escape old solutions and converge stably, and knowing "what changed" is not the same as knowing "how to redesign." Next, self-evolving methods need to turn feedback into falsifiable hypotheses, actively discard outdated experience, and concentrate search resources on mechanisms that are genuinely different and promising. Along this direction, we will release more diverse task types, richer physical mechanisms, and harder tasks, and gradually extend evaluation to richer simulators, real hardware, and longer interaction cycles. As the real world keeps changing and old experience keeps failing, whether an agent can recover autonomously through new interactions may be the ultimate question self-evolution must answer.