arXiv:2608.24087cs.LGcs.AI2026-08

让大模型在推理中主动求助更强模型,提升任务成功率。

Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents

  • 基于贝叶斯最优停止理论,在推理过程中动态判断是否需要升级
  • 实验证明可实现1/√n的后悔率下降,且比事后决策更高效
  • 适合构建自适应的多级大模型代理系统,尤其代码生成场景

当前大模型代理系统要么在推理前决定调用哪个模型(路由),要么在输出后评估并重试(验证)。本文研究第三种模式:代理在推理过程中识别自身可能失败时,主动转移控制权给更强模型。将这一过程建模为基于学习能力后验的贝叶斯最优停止问题——该后验通过标注轨迹学习,而非直接读取原始熵值。推导出贪心式升级阈值的闭式解,通过动态规划刻画最优策略,并证明最优策略是无形状假设下的时变阈值。进一步证明了原信号信念在Chernoff信息率下呈指数分离,后悔界由后验校准决定,且有限样本下:使用n条校准轨迹时,部署的插件策略后悔率以1/√n收敛。受控仿真验证了所有理论预测,包括1/√n速率。真实模型验证在Qwen2.5-Coder 1.5B→7B代码级联任务(MBPP,257任务)上,确认了三项预注册预测中的两项:升级前沿在同等成本下优于事后路由,且累积能力信念的判别力随生成过程上升。

原文摘要 · Abstract (English)

Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior -- an online estimate of the agent's eventual task success whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove that the optimal policy is a time-varying threshold with no shape assumption on the raw signal. We further prove exponential separation of the oracle belief at the Chernoff-information rate of the signal, a regret bound governed by the calibration of the posterior, and a finite-sample guarantee: with n labelled calibration trajectories the deployed plug-in policy's regret decays as 1/sqrt(n). A controlled simulation study confirms each prediction of the theory, including the predicted 1/sqrt(n) rate. We additionally report a real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade (MBPP, 257 tasks), confirming two of three pre-registered predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief's discrimination rises over generation.

大模型代理贝叶斯决策自升级代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。