arXiv:2607.17136cs.SEcs.AI2026-07

修复350亿参数智能体时,单次实验结果不可靠,需多轮验证。

Teach it to stop, not just to click

  • 用验证器引导修复350亿参数计算机使用智能体,分阶段评估可靠性。
  • 单次运行成功率受数据和随机性影响大,最难点中数据差异占48%方差。
  • 修复效果取决于是否为任务唯一障碍,固定动作修复更可靠,适合开发者复现。

在五个由专家评分的环境中,对一个350亿参数的计算机使用智能体(CUA)进行验证器引导的修复。通过方差成分分解(交叉数据抽样×种子网格,置信区间为自助法),发现评估方差可忽略(σ_{eval}≈0),训练种子效应普遍较小(≤10%);主要方差来自数据抽样和运行间非确定性,其中数据抽样在最难任务中占比达48%。该条件下运行分布呈双峰(Hartigan检验p=0.07,k=10),单次运行约30%概率陷入失败模式,均值±标准差不再适用。修复能力分两层:单一固定动作可稳定修复(完成检测0.97±0.06),而开放式修正仅部分有效(空间坐标点击0.53±0.35,生成填充0.14±0.04)。只有当修正动作是任务唯一未解阻塞时,修复才能传递到任务成功(领英任务:修复后8/20,基线0/15,Fisher检验p=0.006)。我们自身两个高调结论(样本效率曲线、'接地无法购买'边界)正是通过多种子复现才被纠正。在类似设置下,单次提升若发表,约三分之一时间会得反向结论。我们开源了工具库cua_reliability,支持常规多种子报告。据我们所知,这是首个在真实350亿参数智能体上实现多模态分段聚合的在线自蒸馏(SA-OPSD)更新。

原文摘要 · Abstract (English)

Agentic computer-use RL is reported in single runs, and those numbers mislead. Using verifier-guided repair of a 35B computer-use agent (CUA) across five oracle-graded environments, we show a repaired policy's success rate is dominated by upstream variance: a variance-components decomposition across three cells (crossed data-draw $\times$ seed grid, bootstrap CIs) finds evaluation variance negligible ($σ_{\mathrm{eval}} \approx 0$) and the training-seed effect small everywhere ($\leq 10\%$); instead it splits between the data draw and run-to-run nondeterminism, the data draw's share rising to dominant ($48\%$) on the hardest cell. There the run-to-run distribution is bimodal (Hartigan dip $p=0.07$, $k=10$), so a single run has roughly a 30% chance of the failure mode and mean$\pm$std is the wrong summary. On that footing, two findings hold. First, repairability is two-tier in how constrained the corrective action is: a single fixed token installs reliably (done-detection $0.97\pm0.06$), while open-ended corrections are only partial -- spatial-coordinate clicks (grounding $0.53\pm0.35$) and a generative field-fill ($0.14\pm0.04$). Second, the frame-level repair transfers to task success only when the corrective action is the task's sole remaining blocker (LinkedIn 8/20 vs. base 0/15, Fisher $p=0.006$). We caught two of our own over-claims -- a sample-efficiency curve and a 'grounding cannot be bought' boundary -- only by replicating across seeds; a stress test makes the stakes external: a single-run improvement of the size this field publishes would have the wrong sign roughly one-third of the time in a comparable regime. We release a library (cua_reliability) for routine k-seed reporting. The apparatus is, to our knowledge, the first multimodal segment-aggregated on-policy self-distillation (SA-OPSD) update on a real 35B CUA policy.

智能体可靠性强化学习验证修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。