提出PACE方法,让自进化智能体更可靠地判断修改是否有效。
PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

- 用赌博式检验实现随时有效的提交判断,避免自我误导
- 在真实改进被噪声掩盖时,错误提交率低于42%,有害修改几乎为零
- 无需训练,可控制每次决策的假阳性,适合追求稳定进化的研究者
自进化智能体通过反复提出自身提示、技能或工作流的修改,并保留那些在小规模预留集上得分更高的版本来提升性能。目前几乎所有努力都集中在生成候选修改的提议者上,而我们指出真正薄弱环节是接受者——决定是否采纳修改的规则。在相同嘈杂验证集上反复应用‘只要得分上升就保留’的规则,本质上是不受控的多重假设检验:智能体相当于自我进行p值操控,累积虚假提交,导致系统漂移而非真正进步。本文将提交过程重新建模为序列假设检验,提出无训练、随时有效的提交门控方法PACE(Paired Anytime-valid Commit Evaluation)。每个候选修改与当前版本在相同实例上对比,仅当测试-下注式e过程积累充分证据时才提交,能提前停止以节省评估成本,并在可选停止条件下将每次提交的假阳性概率严格控制在用户设定水平。在基于Qwen2.5(0.5B-3B)的提示级自进化实验中,贪心接受策略在真实改进被噪声掩盖时,错误提交率达30%-42%,有害修改占比10%-33%;而PACE能准确识别真实改进,几乎不提交其他内容,保持与贪心相当的预留集准确率,方差显著更低,评估成本降低约18%。当无真实提升时,贪心每轮提交13-21次伪修改(72%-100%为假),使最脆弱模型性能下降4.9分;而PACE维持基线不变。自进化可靠性取决于接受者,而不仅是提议者。
原文摘要 · Abstract (English)
Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set. Almost all effort has gone into the proposer that generates candidates; we argue the weak point is the acceptor, the rule that decides whether to commit a change. Applied hundreds of times against the same noisy dev estimate, the ubiquitous "keep it if the score went up" rule is uncontrolled adaptive multiple testing: the agent effectively p-hacks itself, accumulating false commits that make it churn and drift rather than improve. We recast committing as a sequential hypothesis test and propose PACE (Paired Anytime-valid Commit Evaluation), a training-free, anytime-valid commit gate. Each candidate is compared to the incumbent on identical instances and committed only when a testing-by-betting e-process accumulates decisive evidence, stopping early to save evaluations and controlling each candidate's false-commit probability at a user-set level even under optional stopping (a per-decision guarantee). On Qwen2.5 agents (0.5B-3B) self-evolving at the prompt level on GSM8K, SVAMP, and ARC-Challenge, greedy acceptance commits 30-42% false and 10-33% harmful edits when a genuine improvement is hidden among noisy proposals, while PACE commits the real one and essentially nothing else, matching greedy's held-out accuracy at sharply lower variance and about 18% lower evaluation cost. With no real gain available, greedy commits 13-21 spurious self-modifications per run (72-100% false) and degrades the most fragile agent by 4.9 points, while PACE holds at baseline. Reliability of self-evolution depends on the acceptor, not only on the proposer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。