研究大模型工具代理在多步操作中实体绑定错误的演化，发现错误会随时间放大。
Binding Drift in Multi-Step Tool-Augmented Agents
- 区分绑定漂移与错误传播，设计可控测试验证错误演化机制
- 实体锁定策略使错误动作增加3倍，最严重时放大8.5倍
- 用廉价二次验证模型可减少79%错误，接近理想修复效果
工具增强的语言模型代理在外部系统上执行多步工作流，仅需一次实体识别后在后续步骤中重复使用。先前研究表明，在单步操作中，代理会选择正确工具但绑定错误实体的比例为24-26%。本文研究绑定错误随时间的变化：是保持正确、悄然漂移到其他实体，还是从一开始就错误并持续累积？我们明确定义了绑定漂移（初始正确但后期出错）与错误传播（初始即错）的区别，并在互不重叠的工作流集上分别评估二者。在受控的多步测试环境（200个工作流，580个实体绑定评分步骤，四个企业领域，八种模型后端）中发现：(1) 在受控错误注入下，实体锁定（即坚持首次绑定）将错误动作从907次增至2,746次（3.0倍；置信区间[2.8, 3.3]），因它忠实地传递了初始错误；(2) 最受影响模型（Claude Opus 4.5）错误放大达8.5倍；(3) 一种基于LLM的实用再验证机制（单次低成本第二轮模型调用重读原始指令）可使错误动作减少79%（0.21倍；置信区间[0.18, 0.25]），误差仅比理想上限（0.20倍）高1个百分点；(4) 在自然场景（无主动注入错误）下，基线代理在18%的合格工作流中出现绑定漂移，且每步错误率随步骤递增。持久化与再验证不可互换：消除漂移的策略可能加剧传播；而实用再验证几乎达到理想恢复水平。
原文摘要 · Abstract (English)
Tool-augmented language-model agents execute multi-step workflows over external systems, resolving an entity once and then acting on it across subsequent steps. Prior work shows that in single-step actions, agents select the correct tool but bind it to the wrong entity 24-26% of the time. We study what happens to entity bindings over time: do they stay correct, silently drift to a different entity, or, if wrong from the start, propagate and compound? We formalize binding drift (correct at step 1, wrong later) as distinct from error propagation (wrong at step 1, carried forward), and score them on disjoint workflow sets so the two cannot be conflated. In a controlled multi-step testbed (200 workflows, 580 entity-binding-scored steps, four enterprise domains, eight model backends spanning small to frontier), we find: (1) under controlled error injection, an entity lock (the intuitive "persist the first binding" fix) amplifies wrong actions from 907 to 2,746 (3.0x; bootstrap 95% CI [2.8, 3.3]), because it faithfully carries the seeded wrong entity into every later step; (2) the amplification reaches 8.5x on the most affected model (Claude Opus 4.5); (3) a practical LLM-based re-verifier (a single cheap second model call re-reading the original instruction) reduces wrong actions by 79% (0.21x; CI [0.18, 0.25]), closing the gap to within 1 percentage point of an oracle upper-bound (0.20x); and (4) in the natural (non-injected) setting, baseline agents drift on 18% of eligible workflows, with the per-step error rate rising across steps. Persistence and re-verification are not interchangeable: a defense that eliminates drift can worsen propagation, and a practical re-verifier nearly matches oracle recovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。