让大模型学会在工具出错时重试、换路或放弃,提升实际应用鲁棒性。
Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection
- 构建可控故障环境,模拟真实工具失败场景。
- 引入贝叶斯工具记忆,无须重训练提升16.8%鲁棒性。
- 结合强化学习与记忆机制,适合部署于复杂工具链系统。
工具使用型大模型通常在工具调用始终成功的环境中训练和评估,但实际部署中工具可能瞬时、持续或无声失效。鲁棒恢复不仅需要重复尝试,还需判断是否切换路径或终止任务。我们提出BENCH2ROBUST框架,将无故障工具使用基准转化为具有场景化可解性的随机环境,明确要求智能体在任务中执行重试、切换或停止操作。利用该框架,我们研究了两种互补干预:通过贝叶斯工具记忆(BTM)提供结构化运行时恢复上下文,以及课程化强化学习。在4个模型家族的7个模型及两个多轮对话基准上,工具故障均引发普遍性鲁棒性下降。在保留的零售任务上,BTM不需重新训练即可提升16.8个百分点的鲁棒性,而强化学习则学习到互补的恢复行为,即使未使用推理时的BTM仍具价值。两者结合在注入故障下达到40.8%-45.5%的性能,同时保持原无故障表现。结果表明,鲁棒工具使用需融合环境特定恢复知识与学习得来的恢复策略。
原文摘要 · Abstract (English)
Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, persistently, or silently. Robust recovery therefore requires more than repeated retries: an agent may need to retry the same path, switch to an alternative, or recognize that no viable path remains. We present BENCH2ROBUST, a framework that converts failure-free tool-use benchmarks into controlled stochastic environments with scenario-controlled solvability, where episodes explicitly require retrying, switching, or stopping after available paths are exhausted. We use BENCH2ROBUST to study two complementary interventions: structured runtime recovery context through Bayesian Tool Memory (BTM), and curriculum-controlled reinforcement learning. Across 7 models from 4 families and two multi-turn benchmark families, tool failures produce a near-universal robustness gap. On held-out Retail tasks, BTM improves robustness by up to 16.8 percentage points without retraining, while RL learns complementary recovery behavior that remains beneficial without inference-time BTM. Combining the two reaches 40.8-45.5% under injection while preserving failure-free performance. These results suggest that robust tool use benefits from combining environment-specific recovery knowledge with learned recovery behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。