提出可自验证的代理系统,分离长期任务中的承诺漂移与绑定漂移。
The LLM Proposes, the Executive Disposes: A Self-Verifying Agent Instrument that Dissociates Commitment Drift from Binding Drift in Long-Horizon Agents
- 用确定性执行器控制信念,语言模型仅提交带类型提案,通过事前预测验证行为。
- 实验发现移除承诺机制后目标放弃率从0.00升至1.00,而绑定误差保持0.00不变。
- 即使组件被移除,仍能通过影子引用计算漂移指标,适合长期智能体研发验证。
当长期代理的状态与自我报告不可信时,如何验证其行为?我们构建了一种结构性验证的代理仪器:由确定性执行器持有全部信念,语言模型只能提交带类型的提议,且只有当行动前预注册的预测与观测匹配时,主张才被接受。该仪器具备双重自验证特性:任一运行中若违反每器官写入错误、渲染尺寸或盐化哨兵回声阈值,即自动失效(前八次架构运行中有四次因真实缺陷被判定无效);同时,一个渲染不可见的影子引用会编译出全系统在每个消融单元中本应执行的计划,使漂移度量在组件被移除时仍可定义。使用此仪器,我们揭示了所有长期代理均面临的单一变量失败现象:移除承诺机制后,目标放弃率从0.00跃升至1.00,而绑定误差稳定在0.00(三种子种子/单元,每轮最多394个参考基准,所有运行均经有效门控)。相反,当修复绑定通道被消融时,其并未表现为逐周期漂移——因绑定由代码掌控,失败被结构吸收,仅上游表现为假设生成崩溃。我们完整披露任务效能为零(52次门控运行中无任何完成项,基于ARC-AGI-3),此为预先注册的结构性失效条件。核心贡献在于提供一种可测量漂移分解的代理开发验证方法。
原文摘要 · Abstract (English)
How do you verify a long-horizon agent when its own state and self-reports are exactly what you cannot trust? We present an agent instrument built so that verification is structural rather than post-hoc. A deterministic Executive owns all belief; a language model may only file typed proposals, and a claim is admitted only when a prediction pre-registered before acting is matched against observation by code. Two properties make the instrument a verifier of its own science, not just of the agent: every run invalidates itself when per-organ write-error, render-size, or salted-canary-echo floors are breached (four of the first eight architecture runs were invalidated, each localizing a real defect); and a render-invisible shadow reference compiles the plan the full system would have committed in every ablation cell, so drift metrics are defined even where the mechanism under test has been removed. Using this instrument we report a clean, single-variable result on a failure every long-horizon agent suffers: ablating the commitment mechanism flips goal-abandonment from 0.00 to 1.00 while binding error stays flat at 0.00 (three seeds per cell, up to 394 reference beats per run, every run gated valid). The binding channel, by contrast, does not reappear as per-beat drift when its repair is ablated -- because binding is code-owned, the failure class is structurally absorbed, its only residue appearing one layer upstream as a collapse in hypothesis formation. We report these under full disclosure that task efficacy is null (zero level completions across 52 gated runs on ARC-AGI-3), pre-registered as a structural defeater. The contribution is a verification methodology for agent development and the drift decomposition it makes measurable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。