arXiv:2608.21027cs.AI2026-08

用小模型比对选择,实现低成本有效干预大模型决策。

Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents

  • 仅通过比较不同路径优劣决定是否干预,无需求解或批评。
  • 在三个任务上提升全部九项指标,超越现有基线方法。
  • 适合资源受限场景,为轻量级运行时纠错提供新思路。

大语言模型代理正成为处理需要推理、工具使用和序列决策的现实任务的重要范式。随着代理执行时间延长,运行时干预可在不重新训练主模型的前提下提升可靠性。单纯故障检测不足以为继,有效干预还需提供可行的恢复方向。现有方法常依赖专家求解器或具备任务能力的评判模型,带来额外开销或高计算成本。本文提出仅比较型微型顾问(COTA),其通过对比采样路径与原提议的后续表现,判断是否需要干预。该比较器基于同前缀反事实分支构建的成对监督进行训练。优选路径作为非强制性建议返回,由原始代理自主重规划。在WebShop、ALFWorld和tau^3-Retail三个数据集上,使用三种代理进行测试,COTA在全部九个评估场景中均取得改进,并优于所有对比基线。结果表明,即使辅助模型的任务求解能力远弱于主代理,构造性运行时干预依然有效。

原文摘要 · Abstract (English)

LLM agents are emerging as an important paradigm for real-world tasks that require reasoning, tool use, and sequential decision-making. As these agents operate over longer horizons, runtime intervention offers a way to improve reliability without retraining the underlying actor. Failure detection alone is insufficient. Effective intervention must also provide a useful direction for recovery. Existing approaches often rely on an expert solver or a critic that generates task-specific corrections, incurring either the cost of another capable solver or the capacity demands of a task-capable critic. We introduce Comparison-Only Tiny Advisor (COTA), a comparison-only framework for constructive runtime intervention. In COTA, a tiny comparator judges whether sampled alternatives lead to better continuations than the actor's proposal, and repeated comparisons determine when intervention is warranted. We train the comparator using pairwise supervision constructed from same-prefix counterfactual branches. Preferred alternatives are returned as non-binding advice, leaving the original actor to replan. Across WebShop, ALFWorld, and tau^3-Retail with three actors, COTA improves all nine evaluation settings and outperforms the compared baselines. These results show that constructive runtime intervention can remain effective even when the auxiliary model has substantially weaker task-solving capability than the actor.

运行时干预小型模型大模型优化智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。