PilotDeck × Workbuddy Bench full benchmark results

Agent tools are proliferating — which one is more capable?

We put PilotDeck, Claude Code and WorkBuddy together on Workbuddy Bench. Let's take a look at the results.

What we tested:

➀ Four tracks — office, web, code and security — with 260 real tasks; each task run independently three times, for 4,680 formal runs across the four tracks in total

➁ A unified judge model, gpt-5.6-sol; two open-source backbone models: DeepSeek-V4.1-Flash and GLM-5.3

➂ Three frameworks, the same backbone, the same task set — comparing the engineering capability of the frameworks themselves

What each track measures:

➀ Office: In workspaces containing Excel, PDF, Word, CSV and other files, complete data verification, consolidation, file updates and report production, and deliver results that can be checked

➁ Web: Includes building websites from scratch or modifying existing ones, covering page interactions, data visualization, visual design and front-end testing — evaluating both the visual result and the actual operations and state changes

➂ Code: The agent must enter a real open-source project, understand the full repository structure, locate problems across files, modify code, and pass hidden tests

➃ Security: Covers four kinds of work — vulnerability discovery, malware analysis, security operations and agent security — assessing the agent's ability to find problems, provide evidence, complete verification and control risk during testing

Results:

➀ With the DeepSeek-V4.1-Flash backbone, PilotDeck's overall average across the four tracks is 73.66, ahead of Claude Code (73.36) and WorkBuddy (64.56)

➁ In office, web, code and security, PilotDeck scores higher than WorkBuddy

➂ PilotDeck ranks first in several tracks: on web tasks with the GLM-5.3 backbone, PilotDeck scores 68.33, ahead of Claude Code (63.76) for first place; with DeepSeek-V4.1-Flash, PilotDeck ranks first on security with 68.30; on code tasks, PilotDeck scores higher than both Claude Code and WorkBuddy

🔍 Follow us

PilotDeck is jointly developed by THUNLP, OpenBMB and AI9Stars. Its code is open source and not tied to any fixed model — we welcome you to try it and send feedback!

🔗 Website: https://pilotdeck.openbmb.cn/

🔗 GitHub: https://github.com/OpenBMB/PilotDeck