让智能体在执行时实时自我纠错,提升工具调用准确性。
Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents

- 引入评审智能体,在工具调用前进行实时评估,实现主动纠错。
- 在多轮任务中准确率提升7.1%,无关响应检测提升5.5%。
- 提出帮助性与伤害性量化指标,指导评审模型选择与优化。
工具调用智能体在工具选择、参数精度和范围识别方面表现良好,但大语言模型的轨迹评估仍为事后分析。由于脱离执行流程,此类评估只能通过提示调优或重训练修正错误,无法实现实时纠偏。为此,本文将评估引入推理阶段:由专门的评审智能体在执行前对初步工具调用进行评估,推动范式从事后修复转向主动评价与误差抑制。该架构明确分离主执行智能体与评审智能体。如同多智能体系统,评审可能修正错误的同时引入新错误,但此前尚无系统测量此权衡的研究。为此,我们提出“帮助性-伤害性”度量:帮助性衡量反馈纠正的基础智能体错误比例;伤害性衡量反馈破坏正确响应的比例。这些指标直接指导评审设计,揭示模型或提示是否带来净收益。我们在BFCL(单轮)和Tau2-Bench(多轮状态化场景)上评估,实现无关检测+5.5%和多轮任务+7.1%的提升。结果表明评审模型选择至关重要:o3-mini模型的帮助性与风险比达3:1,优于GPT-4o的2.1:1。通过GEPA自动提示优化,进一步提升+1.5%-2.8%。整体证明,执行与评审分离的核心优势在于可独立优化评审模块,无需重训练主智能体。
原文摘要 · Abstract (English)
Tool-calling agents are evaluated on tool selection, parameter accuracy, and scope recognition, yet LLM trajectory assessments remain inherently post-hoc. Disconnected from the active execution loop, such assessments identify errors that are usually addressed through prompt-tuning or retraining, and fundamentally cannot course-correct the agent in real time. To close this gap, we move evaluation into the execution loop at inference time: a specialized reviewer agent evaluates provisional tool calls prior to execution, shifting the paradigm from post-hoc recovery to proactive evaluation and error mitigation. In practice, this architecture establishes a clear separation of concerns between the primary execution agent and a secondary review agent. As with any multi-agent system, the reviewer can introduce new errors while correcting others, yet no prior work to our knowledge has systematically measured this tradeoff. To quantify this tradeoff, we introduce Helpfulness-Harmfulness metrics: helpfulness measures the percentage of base agent errors that feedback corrects; harmfulness measures the percentage of correct responses that feedback degrades. These metrics directly inform reviewer design by revealing whether a given model or prompt provides net positive value. We evaluate our approach on BFCL (single-turn) and Tau2-Bench (multi-turn stateful scenarios), achieving +5.5% on irrelevance detection and +7.1% on multi-turn tasks. Our metrics reveal that reviewer model choice is critical: the reasoning model o3-mini achieves a 3:1 benefit-to-risk ratio versus 2.1:1 for GPT-4o. Automated prompt optimization via GEPA provides an additional +1.5-2.8%. Together, these results demonstrate a core advantage of separating execution and review: the reviewer can be systematically improved through model selection and prompt optimization, without retraining the base agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。