arXiv:2509.18847cs.CVcs.AI2025-09ACL被引 8

让大模型学会从错误中修复,提升工具调用可靠性。

Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions

  • 设计结构化反思机制,自动诊断错误并提出可执行修复方案。
  • 在多轮工具调用中,失败恢复成功率显著提升,冗余调用减少。
  • 适用于需要高可靠性的智能代理系统,如复杂任务规划。

工具增强的大语言模型通常通过监督模仿或粗粒度强化学习训练,仅优化单次工具调用。当前自我反思方法依赖启发式提示或单向推理,模型被要求‘多思考’,但缺乏错误诊断与修复能力。这种做法在多轮交互中脆弱,失败后常重复错误。本文提出结构化反思,将从错误到修复的路径转化为显式、可控且可训练的动作。代理生成简洁精准的反思:基于前序步骤证据诊断失败,并提出正确可执行的后续调用。训练结合DAPO与GSPO目标,采用针对工具使用设计的奖励机制,优化‘反思→调用→最终结果’的分步策略。为评估,引入Tool-Reflection-Bench,一个轻量级基准,程序化检查结构有效性、可执行性、参数正确性和结果一致性。任务以错误调用、反思、修正调用的微型轨迹构建,训练与测试集互不重叠。在BFCL v3和Tool-Reflection-Bench上的实验显示,多轮工具调用成功率和错误恢复能力大幅提升,冗余调用减少。结果表明,显式化并直接优化反思能显著提升工具交互可靠性,并为代理从失败中学习提供可复现路径。

原文摘要 · Abstract (English)

Tool-augmented large language models (LLMs) are usually trained with supervised imitation or coarse-grained reinforcement learning that optimizes single tool calls. Current self-reflection practices rely on heuristic prompts or one-way reasoning: the model is urged to 'think more' instead of learning error diagnosis and repair. This is fragile in multi-turn interactions; after a failure the model often repeats the same mistake. We propose structured reflection, which turns the path from error to repair into an explicit, controllable, and trainable action. The agent produces a short yet precise reflection: it diagnoses the failure using evidence from the previous step and then proposes a correct, executable follow-up call. For training we combine DAPO and GSPO objectives with a reward scheme tailored to tool use, optimizing the stepwise strategy Reflect, then Call, then Final. To evaluate, we introduce Tool-Reflection-Bench, a lightweight benchmark that programmatically checks structural validity, executability, parameter correctness, and result consistency. Tasks are built as mini trajectories of erroneous call, reflection, and corrected call, with disjoint train and test splits. Experiments on BFCL v3 and Tool-Reflection-Bench show large gains in multi-turn tool-call success and error recovery, and a reduction of redundant calls. These results indicate that making reflection explicit and optimizing it directly improves the reliability of tool interaction and offers a reproducible path for agents to learn from failure.

大模型工具调用反思机制错误修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。