AI makes AI: ModelBest open-sources ForgeTrain, the world’s first production-grade training framework written entirely by AI
From May 25 to 29, ModelBest and OpenBMB are co-hosting the On-Device Large Model Open Source Week, unveiling one killer capability of on-device models each day. The peak of on-device models is not just the tip of the iceberg, but the entire iceberg. Today brings the third installment: ForgeTrain, a pre-training framework written entirely by AI.
Can AI write truly production-grade large software — even a large-model training framework?
ModelBest's answer is yes.
Today, ModelBest, together with Tsinghua University and the OpenBMB open-source community, officially releases ForgeTrain — the world's first production-grade large-model training framework written entirely by AI, with zero human code involved.
It is not an experimental prototype but an "AI engine" that can go straight into production and create value.
On NVIDIA H100 GPUs, ForgeTrain trains 10% faster than Megatron, NVIDIA's own mainstream framework. That means 10% of compute costs are saved outright on the same task.
On Huawei Ascend, ForgeTrain has run the full pre-training pipeline end to end, demonstrating strong cross-hardware adaptation.
In addition, ModelBest used ForgeTrain to complete pre-training of MiniCPM5-1B, its new-generation "small but mighty" model, on Huawei Ascend chips. Evaluations show that MiniCPM5-1B leads models of the same size worldwide and ranks Top 1 on the AA leaderboard among models under 2B.
This marks the completion of a full loop: a framework written by AI, running on domestic chips, training a leading AI model.
Behind these results is a new software programming paradigm that we proposed and put into practice: Forge Engineering.
Why must AI make AI?
Every leap in large-model capability has been backed by astronomical resource investment. The cost of a 100,000-GPU cluster is approaching the annual cash flow of top tech companies, new capacity on North America's power grid has been booked up in large blocks by leading labs, and high-quality text corpora are increasingly scarce. If progress in large models can only come from spending more money and using more GPUs, that road is bound to narrow.
This is a foundational question that determines whether large models have a future at all.
A simplified formula describes the growth of large-model capability:
Annual capability growth of large models = Scaling gain per generation × R&D budget / R&D cost per generation
The Scaling gain is bounded by theory, the R&D budget is bounded by resources, and only one lever can be pulled continuously: lowering the R&D cost of each model generation.
This is exactly the starting point of "AI making AI." Leading labs worldwide have made it a strategic priority — Anthropic AI founder Dario Amodei calls it the strongest accelerator for the AGI timeline, and OpenAI and DeepMind are also exploring it at full tilt. Yet at the most critical point — letting AI write the underlying infrastructure for large models — earlier efforts have left a gap.
NVIDIA's VibeTensor, Anthropic's C compiler, and OpenAI's Harness Engineering — three results from leading North American tech companies — each contribute something, but no team has achieved both production-grade usability and outstanding performance at the granularity of "AI writing a training framework."
ForgeTrain makes a decisive breakthrough at precisely this gap. AI did not just write a working training framework; that framework is faster and more performant than what human experts built.
Forge Engineering: from programming to forging
A large-model training framework is not ordinary software. It must orchestrate hundreds or even thousands of GPUs working together, guarantee bit-exact consistency of results in a distributed environment, and squeeze every unit of compute close to its theoretical limit. Writing one is not hard; writing one that works and is also faster is a challenge of an entirely different order.
The Forge Engineering methodology has three stages, each with clear acceptance criteria:
Step 1: Establish the standard (Harnessing)
Collect key runtime data from Megatron and build an automated evaluation harness, turning "runs correctly" and "runs fast" into criteria a machine can judge automatically. Every subsequent AI generation and iteration then has an objective anchor instead of relying on human review.
Step 2: Bit-for-bit alignment (Bit-for-Bit Replication)
Under the harness constraints, the AI builds a version that is bit-for-bit identical to Megatron. Bit-for-bit identical does not mean roughly aligned or behaviorally similar; it means producing exactly the same numerical results from the same inputs. This step establishes the framework's correctness baseline: the AI can precisely understand and reproduce a complex system.
Step 3: Remove constraints and surpass (Surpassing)
Once correctness is verified, the binary-identity constraint is lifted and the system switches to a new performance-oriented harness, letting the AI iterate autonomously in a much larger search space. The final result: 10% faster than NVIDIA Megatron, saving 10% of compute costs.
This three-stage path — reproduce bit for bit first, then relax the constraints and surpass — was absent from all previously published methodologies.
VibeTensor pursued behavioral alignment with PyTorch directly but lost some performance; Anthropic's C compiler used GCC for differential testing but stopped at correctness verification rather than surpassing on performance; OpenAI's harness had no reference implementation to align against. ForgeTrain is the first team to set out these three steps explicitly and deliver on all of them.
This three-stage path points to an entirely new software engineering paradigm, which we call Forge Engineering.
To understand its significance, consider how programming paradigms have evolved in the large-model era:
OpenAI's Harness Engineering was an important milestone: it proved that AI can generate large general-purpose systems end to end. But code is still treated as an asset that must be maintained over the long term, with a main branch, a release cadence, and steadily accumulating technical debt.
Forge Engineering crosses the final threshold. It unties code from being an "asset" and turns it into a deeply customized artifact forged on demand — the same harness can forge one implementation on H100 and a completely different one on Ascend. Both are deeply optimized for their own hardware, with no performance penalty paid for generality.
Large-scale software engineering once faced an unresolvable trade-off: generality requires abstraction, and abstraction costs performance; efficiency requires customization, and customization sacrifices portability. Forge Engineering makes that trade-off disappear. Generality stays in the harness specification; high performance stays in each act of forging.
Reshaping the place of domestic compute in the AGI era
Jensen Huang has repeatedly said that "NVIDIA is fundamentally a software company." What truly makes customers unable to leave NVIDIA is not the chips but the CUDA ecosystem built over more than a decade by millions of developers. That is NVIDIA's deepest moat.
Domestic chips have advanced rapidly in hardware, but the software ecosystem remains the biggest pain point. The core challenge: making every algorithm run efficiently on every chip. The combination space is astronomical, and human effort can never catch up.
Forge Engineering changes how the problem is solved:
When the marginal cost of writing code approaches zero, we no longer need to maintain a universal framework that tries to support every chip. Instead, we maintain a set of specifications and evaluation standards (the harness) and forge a dedicated high-performance implementation on the spot for each chip and each scenario. When a chip generation turns over or a model iterates, we forge a new one.
This means the vicious cycle in which every new generation of domestic chips forced the software ecosystem to catch up again with human labor may be broken for good. We no longer need to spend a decade building another CUDA; instead, AI forges a dedicated software stack for each chip on demand. However fast chips ship, software can keep pace. The logic of the moat has changed: it is no longer about who piles up more code, but whose "forge" is faster and more accurate.
Compute can be blockaded; engineering efficiency cannot. ForgeTrain is the first solid proof on this path.
We divide the evolution of "AI making AI" into five levels, L1 through L5, echoing OpenAI's L1-L5 framing for AGI. ForgeTrain corresponds to L3: AI produces, end to end in a closed loop, the infrastructure needed for the next generation of large models:
Open source and reproducible
One sentence captures the strategic significance of "AI making AI":
Once the rate at which AI researches AI exceeds the rate at which humans research AI, the pace of large-model R&D shifts from a function of headcount to a function of compute, and the timeline for AGI will be rewritten.
MiniCPM5-1B, pre-trained with ForgeTrain, achieves the best performance among models of its size worldwide, demonstrating that "AI making AI" is feasible.
ForgeTrain is fully open source. We are releasing, together with it:
ForgeTrain framework code: the complete pre-training framework, forged in two versions for NVIDIA H100 and Huawei Ascend.
Agent Harness: the complete toolchain that produced ForgeTrain, including evaluation standards, test specifications, and agent orchestration. Any team can use it to reproduce the entire process from bit-for-bit alignment to performance surpassing.
What we open-source is not just code but a methodology. From the efficient on-device models of the MiniCPM series to today's ForgeTrain and Forge Engineering, the core question we explore has never changed: how do we make every unit of compute deliver more value?
The answer is growing clearer: it is not about who has more resources, but who uses them more intelligently.
When AI learns to forge its own tools, the ceiling on efficiency is broken once again.
➤ ForgeTrain on GitHub
🔗 https://github.com/OpenBMB/ForgeTrain
If you would like to join ModelBest's "AI making AI" team, send your resume to career@modelbest.cn.