不同语言模型的推理策略应因模型多样性而异,盲目探索或精炼都可能无效。
Know Thy Reasoner: Not All Language Models Explore Alike

- 根据模型解题思路的分布差异,选择深挖或广搜策略
- 低多样性模型靠轻量信号深挖更有效,高多样性模型反而受损
- 揭示了模型推理失败的关键原因,适合研究大模型推理机制者阅读
大语言模型推理中的计算扩展在探索新解法(广度)与深化已有方案(深度)之间存在权衡,但为何某种权衡有效、为何难以迁移仍不清晰。我们提出,最优策略取决于模型的‘多样性特征’——即概率质量在不同解法路径上的分布情况,必须先刻画该特征再决定探索方式。我们构建了一个分解推理不确定性的框架,推导出深度优化优于并行采样的条件,并在三种模型族中验证了该理论,涵盖推理与训练阶段。核心发现是:多样性水平决定策略选择——低多样性对齐模型适合通过轻量内在信号进行深度优化;而高多样性基础模型常被深度策略损害,需依赖广度探索或更强信号来补偿。
原文摘要 · Abstract (English)
Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why it often fails to transfer across models, remains unclear. We argue that \textbf{the optimal strategy depends on the model's \emph{diversity profile}, the spread of probability mass across solution approaches, and that this must be characterized before any exploration strategy is adopted.} We formalize this with a framework decomposing reasoning uncertainty, deriving when depth-based refinement outperforms parallel sampling, and validate it across three model families at both inference and training. Our central finding is that the diversity regime dictates the strategy: low-diversity aligned models benefit from depth-based refinement with lightweight intrinsic signals, whereas high-diversity base models are often harmed by it, and instead need breadth or stronger signals to compensate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。