修复对话历史误导下的工具调用错误,提升大模型决策可靠性
When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories
- 设计三视图对比基准,分离原始、污染与黄金状态,定位错误根源
- 在Qwen3-1.7B上实现87.0%准确率,优于现有方法30%以上
- 可迁移至不同场景,对噪声和新功能保持鲁棒性
工具调用智能体依赖对话历史推断任务状态。但在持续交互中,历史记录可能在结构与语义上仍合理,却已不再适用于当前请求。我们发现,此类污染历史会劫持模型原有策略:在Qwen3-1.7B上导致32.1%的正确决策发生偏移,频繁引发被污染实体或接口约定的重复使用。为此提出bench,一个包含原始、污染与黄金状态三视图的配对基准,保留系统策略、当前工具、最新请求及正确下一步动作。十一种保持黄金答案的干预手段揭示了决策状态、实体绑定与接口执行中的失败模式。进一步提出ours方法,通过软监督将仅观察污染历史的学生模型,从条件于黄金状态的教师模型中学习。在Qwen3-1.7B上,ours达到87.0%平衡工具调用准确率,优于黄金SFT(66.3%)、黄金序列蒸馏(82.3%)和离策略词蒸馏(85.0%)。该方法具可扩展性:8B教师使1.7B学生准确率升至91.9%,8B学生达93.0%。所获策略还可迁移到干净历史、未见函数、独立重生成评估环境、外部工具调用基准及噪声多跳问答任务。结果确立历史可靠性为工具调用的独立瓶颈,并验证可靠状态策略迁移的有效性与可扩展性。
原文摘要 · Abstract (English)
Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain structurally valid and semantically plausible after they cease to be authoritative for the current request. We show that such history can hijack a policy the model already possesses: on Qwen3-1.7B, pollution flips 32.1% of decisions that are correct under the original trajectory and frequently induces reuse of corrupted entities or interface conventions. We introduce bench, a paired benchmark with synchronized Original, Polluted, and Oracle State views that preserve the system policy, current tools, latest request, and gold next action. Eleven gold-preserving interventions isolate failures in decision state, entity binding, and interface execution across complete calls and non-call decisions. We further propose ours, which transfers an Oracle-conditioned teacher policy to a student observing only polluted history through soft supervision on student-generated prefixes. On Qwen3-1.7B, ours achieves 87.0% Balanced Tool-Use Accuracy, outperforming Gold-SFT (66.3%), Oracle sequence distillation (82.3%), and off-policy token distillation (85.0%). The method scales consistently: an 8B teacher raises the same compact 1.7B student to 91.9%, while an 8B student reaches 93.0%. The resulting policies further transfer to clean histories, unseen functions, independently regenerated evaluation contexts, external tool-use benchmarks, and noisy multi-hop question answering. These results establish history reliability as a distinct tool-use bottleneck and demonstrate reliable-state policy transfer as an effective and scalable solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。