提出新方法RPC,让大模型推理更准更省资源。
A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
- 结合困惑度与自一致性,提升推理路径筛选效率。
- 实测在7个数据集上推理误差显著降低,采样成本减半。
- 适合追求高可靠性推理且资源受限的AI研究者。
测试时扩展通过增加计算资源提升大语言模型推理性能。主流方法为基于采样的测试时扩展,通过生成多个推理路径来增强结果。然而其理论基础尚不明确。本文首次构建基于置信度估计的理论框架,分析自一致性与困惑度两种主流范式,揭示自一致性存在高估计误差,而困惑度有显著建模误差及估计误差收敛可能退化的问题。为此提出RPC,融合两个关键组件:困惑度一致性与推理剪枝。困惑度一致性将估计误差收敛从线性提升至指数级,同时保持建模误差;推理剪枝通过剔除低概率路径防止性能退化。理论与实证均表明,RPC在7个基准数据集上显著降低推理误差,性能媲美自一致性,但采样成本减少50%,且置信度更可靠。代码与资源已公开。
原文摘要 · Abstract (English)
Test-time scaling seeks to improve the reasoning performance of large language models (LLMs) by adding computational resources. A prevalent approach within the field is sampling-based test-time scaling methods, which enhance reasoning by generating multiple reasoning paths for a given input during inference. However, despite its practical success, the theoretical foundations remain underexplored. In this paper, we provide the first theoretical framework for analyzing sampling-based test-time scaling methods, grounded in the perspective of confidence estimation. Based on the framework, we analyze two dominant paradigms: self-consistency and perplexity, and reveal key limitations: self-consistency suffers from high estimation error while perplexity exhibits substantial modeling error and possible degradation of the estimation error convergence. To address these limitations, we introduce RPC, a hybrid method that leverages our theoretical insights through two key components: Perplexity Consistency and Reasoning Pruning. Perplexity Consistency combines the strengths of self-consistency and perplexity, boosting the convergence rate of estimation error from linear to exponential while preserving model error. Reasoning Pruning prevents degradation by eliminating low-probability reasoning paths. Both theoretical analysis and empirical results across seven benchmark datasets demonstrate that RPC has a strong potential for reducing reasoning error. Notably, RPC achieves reasoning performance comparable to self-consistency while not only enhancing confidence reliability but also reducing sampling costs by 50%. The code and resources are available at https://wnjxyk.github.io/RPC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。