arXiv:2608.23651cs.SEcs.AI2026-08

小模型代理会重复失败的指令,因为错误信息反而让模型更想重试。

Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail

论文配图:Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail
图 1 · 摘自论文原文
  • 用错误信息作为反馈,反而降低模型重试失败指令的概率,但实际效果是反向增强。
  • 在测试中,重复失败指令的概率从6%升至54%,令牌级重复率达19%。
  • 修复方法:用运行时生成的失败描述替代原指令,可减少76%的错误重复。

代理记录失败的工具调用及其错误信息,并假设错误信息具有修正作用。我们评估了这一假设的有效性。将失败记录的修正增益定义为重新发出失败动作的对数概率变化,发现所有测试的指令微调模型(6个检查点,135M-1.7B参数,4个模型家族)在两种环境(模拟工具调用与MBPP程序修复)下该增益均为负值。归一化后,每动作令牌的效应约为-1.03纳特,相当于每个令牌的几率下降2.8倍,且在90%-100%的个体样本中成立。固定候选集下,重复失败调用的概率从0.06上升至0.54;贪婪解码在失败后以19%的频率逐令牌重试,而失败前为0%。通过对照实验分离出表面形式与语义影响:失败调用的原始形式贡献了83%的负面影响,而标记为失败的语义影响较小且在不同环境中符号不一致。问题出在代理框架而非模型理解错误信息的能力,这预示了有效修复方案。将原始调用替换为运行时生成的失败描述,可消除76%的反向效应,且无额外令牌开销;使此前失败的字符串在解码器中不可达也产生相同效果。两个常见做法无效:明确“不要重复”指令未改变测量结果;删除失败尝试以避免上下文污染,反而最差,因为它恢复了导致失败的上下文。

原文摘要 · Abstract (English)

Agent harnesses record a failed tool call and its error message in the transcript and ask the model to continue, on the assumption that the error is corrective information. We measure whether it is. Defining the corrective gain of a failure record as the change in log-probability of re-emitting the action that just failed, we find the gain is negative for every instruction-tuned model we tested (6 checkpoints, 135M-1.7B, 4 families) in two environments: simulated tool calling and MBPP program repair. Normalised by action length the effect is about -1.03 nats per action token, a factor of 2.8 in the odds of each token, and holds on 90%-100% of individual items, not only on average. Over a fixed candidate set the probability of repeating the failed call rises from 0.06 to 0.54, and greedy decoding reproduces it token for token on 19% of items after the failure versus 0% before. Counterfactuals pairing the same call with a failure message, a success message, or a neutral acknowledgement separate two effects: the failed call's surface form accounts for 83% of the damage, while the semantic contribution of marking it failed is small and inconsistent in sign across environments. The problem is in the harness, not the model's grasp of error messages, and that predicts which remedies work. Replacing the verbatim call with a runtime-generated description of the failure removes 76% of the inversion at no token cost, and making previously-failed strings unreachable at the decoder acts on the same term. Two plausible remedies do not: an explicit "do not repeat" instruction leaves the measured quantity where it was, and deleting the failed attempt to retry from a clean context, the standard prescription for context contamination, is the worst harness we measured for repetition, because it restores the context that produced the failure. The study runs end to end on a CPU; all artefacts are released.

大模型错误反馈行为偏差调试机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。