编码代理常自信出错，需用验证正确率评估可信度。
Confident and Wrong: Silent Semantic Failures in Coding Agents
- 通过重复运行测试，区分提交率与验证通过率
- GPT-5提交率100%但仅44%任务真正解决
- 发现“无声语义错误”是主要失败模式，且难以被检测
随着编码代理进入生产流程，团队不仅需要知道代理是否完成任务，还需判断其行为是否可信。我们发现完成度与可信度存在显著且系统性的偏离。在50个SWE-bench Verified任务上，对四个前沿模型进行1,750次轨迹测试，分离提交率与测试验证通过率。GPT-5提交率为100%，但仅44%任务成功解决；Llama 4提交率99%，解决率仅为18%；而Gemini提交率最低（70%），解决率却达50%，高于GPT-5。这些差距并非随机，而是集中于一种危险的失败模式——无声语义错误。在有缺陷的任务上，代理在五次运行中均提交看似合理的补丁，但无一通过，重复相同的误解而非随机错误。定量分析显示，该模式占Llama 4失败轨迹的80%、GPT-5的68%。该错误不可见：结果一致且自信地错误，导致基于完成率和一致性监控的评估误判。轻量级预编辑提示无法缩小差距。另一项探测发现，即使在已修复的漏洞上，多数模型仍会修改正确代码，体现出“行动偏好”——在无需操作时仍强行干预。这正是完成率指标所奖励的行为。核心问题在于度量：提交率反映行为，可信度依赖有效性。因此评估必须更新：以多次运行下的测试验证正确率评分，报告不确定性，并奖励那些知道何时不行动的代理。
原文摘要 · Abstract (English)
As coding agents move into production workflows, teams need to know not only whether an agent completes a task, but whether its action can be trusted. We show that completion and trustworthiness diverge sharply and systematically. Across 1,750 trajectories on 50 SWE-bench Verified tasks, we compare four frontier models over repeated runs and separate submit rate from test-verified resolve rate. GPT-5 submits a patch on 100% of runs but resolves only 44%; Llama 4 submits on 99% but resolves 18%; and Gemini, despite submitting least often at 70%, resolves more tasks than GPT-5 (50% versus 44%). These gaps are not random: they concentrate in one dangerous failure mode we call silent semantic failure. Qualitatively, on a buggy task the agent submits a plausible-looking patch on all five runs, yet none pass, the same misinterpretation repeated rather than random error. Quantitatively, it dominates failure, covering 80% of Llama 4's failing runs and 68% of GPT-5's, and it is invisible: the outcomes are confidently and consistently wrong, so completion-based and consistency-based monitoring both look healthy exactly when the agent should not be trusted. Lightweight pre-edit prompts do not close the gap. A second probe isolates the instinct to act: given an already-fixed bug, where the right move is to abstain, most models still edit the correct code. This action bias, acting when no action is warranted, is exactly what completion metrics reward. The throughline is measurement: submit rate captures action, but trust requires validity. So evaluation must catch up: score agents by test-verified correctness over repeated runs, report its uncertainty, and reward those that know when not to act.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。