Tsinghua Team Presents AutoSFT: The First Step Toward Automated Post-Training Infrastructure
This article was produced by researchers from Tsinghua University in collaboration with Beijing University of Posts and Telecommunications, ModelBest, the University of Illinois Urbana-Champaign, and The Chinese University of Hong Kong.
What happens when you hand every SFT data-research operation to one agent and let it run for dozens of rounds?
By any reasonable expectation, an agent with a full toolbox should try different kinds of data operations: rebalancing the mixture, repairing samples, writing new data. That is how an experienced data engineer works.
What actually happened was something else: every single round, the agent did nothing but rebalance the mixture. It never edited a line of data, never replaced a single response, never wrote one new sample. Its word cloud was almost entirely weights and proportions, and dozens of rounds produced little improvement.
Left: benchmark scores by round. Right: the words the agent used to describe its data operations — almost all of them about mixture ratios.
This is what Tsinghua University's THUNLP observed in AutoSFT. The team built a multi-agent system for SFT data research, in which the agents decide for themselves what data operation to perform each round: retrain from the base model, read the evaluation results, decide the next round's data action, then move on to the next round. Along the way, the single-agent baseline used only one of the six data actions, while the multi-agent version used all six — and led the single-agent baseline by 6.5 points on instruction following (IFEval, compared at equal round counts).
Paper cover page: AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Background: AI has already started building AI
Anthropic reports that, as of May this year, more than 80% of the code merged into its production codebase was written by Claude; OpenAI reports that by mid-August, the runtime of agents in its research department, converted into 8-hour workdays, was already 3.1 times that of its human researchers. Writing code and running experiments are increasingly being handed to agents.
And training the models themselves? Early programmers dealt with hardware directly in machine code of 0s and 1s; later, operating systems abstracted that layer away. Training a model today still looks a lot like that earlier stage: tweak the config, watch the curves, dig through bad cases, adjust the data, rerun — every step needs a human.
What the team wants to build is an RSI Service — the "operating system" for AI for AI: you express a goal in natural language, and the system designs the experiments, executes them, analyzes the results, and iterates, so that people no longer have to deal with code frameworks directly. The RSI loop needs an intelligent brain to control the whole process; whether to change the weights, the harness, the algorithm, or the data are all just means to the goal — they can even be opaque to humans.
In PostTrainBench, we found that when agents are given control of the entire post-training pipeline, they all converge on SFT and spend most of their effort on changing data. So, as the first module of the RSI Service, AutoSFT places its action space on SFT data. Data research is also research enough: form a hypothesis, intervene, look at the result, and correct course.
One iteration loop of AutoSFT
Six kinds of data actions, and the single agent used one
An experienced data engineer does more than pick data. The team sorted SFT data-research operations into six categories: Mixture, Ordering, Pruning, Repair, Resupervision, and Synthesis.
The six data actions and how far each one alters the recipe
The team handed all six operations to a single agent and let it run for 11.5 hours and 33 rounds. Every round, it only rebalanced the mixture. Data decisions, job submission, and evaluation were all crammed into the same session; the context window peaked at 94.7% occupancy, was compacted 10 times, and the research history was repeatedly discarded. The biggest space hogs were training output, run logs, and evaluation output — not one of them about "what data operation to do next." Shell commands made up 58% of logged events, 268 of which were status checks, 23.0% of all events.
Context usage of the single agent; dashed lines mark each compaction
When we set the optimization objective to "as many iterations as possible within 10 hours," the agent has no idea what "good" actually means, so it sometimes stops early. It also gets anxious about time: when little time is left, it tends to shrink each round's data volume and leave part of the data budget unused, all to squeeze in more experiments — but from a human perspective, shrinking the data volume makes the results incomparable.
In the later rounds, per-round training data volume plummeted simply in order to "run more experiments"
AutoSFT: splitting the data-research lifecycle into roles
Overview of the AutoSFT framework
The team's solution was to split the lifecycle apart. An RSI Service has to be backed by a cluster, and running experiments means continuously submitting jobs to it, so confirming job status and detecting reward hacking are polling tasks. We carved out an Executor that handles training and evaluation on its own and returns only artifact paths, no longer stuffing logs into the main session.
With vs. without the Executor; with it, the compaction count drops (middle)
Compactions fell from 10 to 5, and the share of shell commands dropped from 58% to 29% — and a third of what remains is still sub-agent scheduling.
Then came the split on the decision side. Inspired by Anthropic's AAR, we have four Analysis Subagents offer recommendations from four angles — training data, failure cases, learning dynamics, and research history — with each sub-agent seeing only the slice of context it needs. A Synthesis Subagent writes new data for capabilities the pool does not cover, with a verifier as gatekeeper. The meta agent aggregates everything and decides the next round's recipe.
On instruction following, with Analysis added, round 6 already reached 63.33, beating the single-agent baseline's best result across 50 rounds; compared at equal round counts, AutoSFT leads by 6.5 points. By round 30 it reaches 67.10.
Left: Analysis ablation, IFEval score by round. Right: how often each action family was used over 30 rounds
By action usage: the single-agent baseline was all Mixture across its 33 rounds. In AutoSFT's 30 rounds, all six action types appear, in a fairly even distribution.
On the other hand, the Synthesis Subagent writes new data for capabilities the pool does not cover. On IFEval and IFBench, Synthesis brought gains of +3.4 and +2.9 (81.58 vs 78.15, 51.95 vs 49.02), with a peak of 65.8.
Left two panels: Synthesis ablation on instruction following. Right: AutoSFT performance in the coding domain
On code generation (MBPP+), the single-agent baseline ran 15 rounds without sustained improvement, while adding Analysis produced +3.7 within the same number of rounds. MBPP+ only runs test cases and provides no labeled capability taxonomy, which shows that Analysis does not depend on a ready-made taxonomy — it can surface problems from failure cases and training dynamics.
The paper also shows per-capability Synthesis results. repeat_prompt jumped from about 40% to 75.61% after the Synthesis rounds — a capability that Synthesis clearly filled in. Some gaps remain: odd_even_syllables was targeted by Synthesis for three rounds and stayed at 0. Supplementary data can probably only fill in capabilities the model is "almost able to do."
Per-capability effect of Synthesis
Conclusion: scope and practical implications
The action space facing AI for AI is enormous: change the model weights or change the harness? optimize the algorithm or optimize the data? Most existing work focuses on the full post-training setting and hands autonomy to frontier agents. In AutoSFT, the first step of the RSI Service, we focus on the data action space in the SFT setting — and even there we found that models struggle to find good combinations of action traces.
The bottleneck in automated SFT data research may not be that some action is too weak, but whether the structure lets an agent cover the full action space of a data engineer. AutoSFT is still far from "train a model with one sentence," but it is a workable answer we can offer.
📄 Paper: arXiv 2610.22194
💻 Code: GitHub Thinking-Space/AutoSFT