arXiv:2607.24300cs.CLcs.MA2026-07

自验证不可靠,需外部审计才能防止模型自我退化。

Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents

论文配图:Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents
图 1 · 摘自论文原文
  • 引入外部固定审计机制,阻止智能体自我欺骗性改进
  • 弱模型易破坏已有能力,强模型仍误判真实表现
  • 适合追求安全自进化系统的研究人员和开发者

自进化智能体通过反复重写规则或策略提升能力,通常依赖自编测试判断是否采纳新版本。由于智能体同时控制优化目标与验证器,自评分数可能始终很高,而实际部署性能却下降或停滞。本文研究验证器-部署差距问题,提出密封外部接受环(SEAL):保留自编测试,但每次候选方案与当前版本通过固定外部审计对比,智能体无法访问或修改审计过程,仅接收接受/拒绝反馈,且在明显退化时保留原状态。实验表明该问题常见于基于试错的启发式学习场景,自验证失败随能力水平分层:弱模型易破坏已有策略,强模型更稳定但仍误判部署分布。标准自约束无法可靠弥合差距。相比之下,SEAL在六个模型和三个随机种子下均优于无保护基线。可靠自进化无需放弃自验证,但必须至少有一个不受智能体控制的部署接受信号。

原文摘要 · Abstract (English)

Self-improving agents accumulate capability by repeatedly rewriting procedural policies, controllers, or heuristic rules. They typically rely on self-authored tests or metrics to decide whether to accept subsequent edits. The agent controls both the optimized object and its verifier. As a result, self-assigned scores can remain near perfect while real deployment performance degrades or stays low. We study this problem through the verifier--deployment gap. This gap refers to the discrepancy between an agent's self-authored verification signal and a sealed deployment evaluation that the agent cannot observe or access. We ask how self-authored verification fails under iterative policy-and-test rewriting, how the failure changes with capability, and how little exogenous trust is sufficient to prevent real regressions from being deployed. To address this problem, we introduce a Sealed Exogenous Acceptance Loop (SEAL). SEAL retains self-authored tests but compares each candidate with the incumbent through a fixed harness-side audit. The agent cannot author or inspect the audit, receives only accept/reject, and the whole incumbent state is retained after a clear regression. Our experiments show that this problem often appears in heuristic learning settings. These settings require trial-and-error discovery of the target objective. We further find that failures of self-written verification are stratified by capability. Weaker agents tend to damage previously acquired strategies behind easy self-tests. Stronger agents are more stable, but they still mismeasure the deployment distribution. Standard self-written constraints do not reliably close this gap. In contrast, SEAL outperforms unprotected baselines across six models and three random seeds. Reliable self-improvement need not abandon self-verification, but it requires at least one deployment-acceptance signal outside the agent's control.

自进化验证漏洞外部审计智能体安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。