测试并提升大模型长任务中持续完成定量目标的能力
Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents
- 设计基准测试PushBench，量化评估代理在收集指定数量成果时的持续性
- 状态追踪控制器达69%-78%成功率，杜绝重复提交，显著优于传统方法
- 发现定量目标需持续验证进度，仅局部任务正确无法保证最终成功
长时序语言代理虽能做出大量看似合理的工具调用，却常无法坚持完成指定数量的目标。本文研究这一差距，提出量化目标持久性（QGP）：代理是否持续工作直至外部验证器确认足够数量的有效独立成果。PushBench将此转化为基准测试，用于仓库-制品收集和验证器支持的工作单元，直接测量重复工作、重复提交、虚假完成和进度漂移，而非依赖最终成功标志。在匹配控制器对比中，状态追踪检索控制器达到69%-78%成功率，且完全消除重复提交；而队列追踪工作单元控制器在标准与完成门控控制器均无法完成任务的场景下，实现25%-50%成功率。黑盒前沿代理评估显示，Claude Code（Sonnet 4.6）和Codex CLI（gpt-5.4）可解决多数50制品任务，但在100制品任务中成功率降至每条件3/9。结果表明，定量目标对可靠性提出了不同于局部任务能力的新要求：代理必须维持经验证的进度，并仅在请求工作完全完成后停止。
原文摘要 · Abstract (English)
Long-horizon language agents can make many plausible local tool calls yet fail to persist until a requested count is actually complete. We study this gap as Quantitative Goal Persistence (QGP): whether an agent keeps working until an external verifier confirms enough distinct valid items. PushBench turns this into a benchmark for repository-artifact collection and verifier-backed work units, so repeated work, duplicate submissions, false completion, and progress drift are measured directly rather than hidden behind a final success flag. In matched controller comparisons, a state-tracking retrieval controller reaches 69-78% success while eliminating duplicate submissions, and a backlog-tracking work-unit controller reaches 25-50% success in settings where standard and completion-gated controllers complete no task instances. Black-box frontier-agent evaluations with Claude Code (Sonnet 4.6) and Codex CLI (gpt-5.4) solve many 50-artifact tasks but drop to 3 out of 9 successes per condition at 100 artifacts. The results show that quantitative goals stress a different reliability requirement from local task competence: agents must maintain verified progress and stop only when the requested work is complete.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。