让大模型推理更可靠，通过量化不确定性避免误删正确路径
Robust Search with Uncertainty-Aware Value Models for Language Model Reasoning
- 用值函数分布代替单一数值，衡量推理结果的可信度
- 在未见推理路径上提升解题覆盖率，尤其在困难题目中表现更好
- 适合追求高鲁棒性的大模型推理系统开发者使用
值模型引导的搜索能有效指导大模型生成，但存在鲁棒性不足的问题，根源在于验证器失败：不完美的值模型会错误地剔除有效的推理路径，尤其是在搜索过程中生成的未见过路径上。为此，本文提出一种不确定性感知框架，包含两个关键组件：(1) 不确定性感知值模型（UVMs），将单点值估计替换为值分布，以量化预测可靠性；(2) 分组汤普森采样，一种基于成为最优候选概率的高效选择算法。在两个分布内（ID）设置（GSM8K、MATH）和三个分布外（OOD）设置（如AIME25、Minerva Math）上的实验表明，该方法显著缓解了验证器失败问题，提升了解题覆盖率，尤其在OOD任务中效果突出。本工作首次系统性地将不确定性量化引入大模型搜索范式，增强了整体鲁棒性。代码已开源：https://github.com/FreedomIntelligence/UVM。
原文摘要 · Abstract (English)
Value model guided search is effective in steering LLM generation but suffers from a lack of robustness. This is due to verifier failure: imperfect VMs mistakenly prune valid reasoning paths, especially when encountering unseen reasoning paths generated during search. To address this, we propose an uncertainty-aware framework with two key components: (1) Uncertainty-Aware Value Models (UVMs), which replace single-point value estimates with value distributions to quantify prediction reliability, and (2) Group Thompson Sampling, an efficient algorithm that selects candidates based on their probability of being optimal. Experiments on two In-Distribution (ID) settings (GSM8K, MATH) and three Out-Of-Distribution (OOD) settings (e.g., AIME25, Minerva Math) show our method significantly mitigates verifier failure and boosts solution coverage, especially on OOD problems. This work provides the first systematic integration of uncertainty quantification into LLM search paradigms, enhancing robustness. The code is released at https://github.com/FreedomIntelligence/UVM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。