arXiv:2605.06908cs.LGcs.AI2026-05被引 3

让大模型智能判断何时该多算,避免反效果。

Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents

论文配图:Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents
图 1 · 摘自论文原文
  • 根据状态特征学习计算是否有益,动态调整决策方向。
  • 在六种环境、三种模型上均优于固定方向的现有方法。
  • 适合需要灵活调用算力的复杂推理任务场景。

针对大模型代理的自适应测试时计算,现有方法通常基于置信度、不确定性或难度设计门控机制,假设信号与计算需求方向一致。本文发现这种方向一致性不稳定:同一信号在不同环境或模型中可能预测计算带来收益或损害,甚至在同一任务下也出现反转。这反映出计算需求与计算适用性之间的本质区别——高不确定性可能表示需要回溯评估的困难状态,也可能表示不适合通过回溯改进的当前情境。固定方向的门控在异构环境下不可靠。为此,提出方向感知自适应学习(DIAL),通过无信号偏见的反事实探索训练稀疏门控,学习状态特征在特定环境与模型下的计算效用方向。在六个环境和三个骨干模型上,DIAL 均实现更优的成功率-成本权衡。

原文摘要 · Abstract (English)

Adaptive test-time compute for LLM agents aims to invoke extra computation only when it improves performance. Existing methods typically use confidence-, uncertainty-, or difficulty-based gates, assuming a fixed direction from the gating signal through compute need to the value of computation. This makes gating a utility-calibration problem: gating signals should align with whether extra computation improves the final outcome over the base policy. We show that this alignment is unstable: the same signal predicts rollout benefit in one setting and rollout harm in another, with reversals across environments and backbones even when the task is fixed. Wrong-direction gates can therefore worsen performance by precisely selecting harmful states. This reversal reflects a deeper distinction between compute need and compute suitability: a high uncertainty signal may indicate decision-difficult states where rollouts help compare alternatives, or intervention-unsuitable states where the current context does not support useful rollout-based improvement. Under this two-source model, fixed-direction gates are unreliable across heterogeneous settings. To address this, we propose DIAL (Direction-Informed Adaptive Learning), a sparse gate trained from signal-agnostic counterfactual exploration to learn the utility direction of state features per (environment, backbone). Across six environments and three backbones, DIAL yields a stronger overall success-cost trade-off than fixed-direction baselines.

大模型代理自适应计算门控机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。