提出RPC方法,让大模型推理更准更快更省资源。
Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning
- 结合困惑度与自一致性,用剪枝剔除低概率推理路径。
- 在7个数据集上显著提升推理准确率和样本效率。
- 适合需要高效可靠推理的AI应用开发者。
大型语言模型在复杂推理任务中表现出色,但单次推理常不可靠。现有方法如困惑度和自一致性存在误差缺陷:前者因缺乏一致性函数导致模型误差大,后者因误差收敛慢导致估计误差高。本文首次对这些方法进行理论误差分解分析,并提出推理剪枝困惑度一致性(RPC)方法。该方法融合困惑度一致性与推理剪枝机制,通过剔除低概率推理路径,有效抑制估计误差恶化。理论分析表明,RPC可使估计误差收敛速度达到指数级,并具备降低模型误差潜力。在七个基准数据集上的大量实验证明,RPC显著提升了推理性能、样本效率和置信度可靠性。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have demonstrated remarkable reasoning capabilities. However, single-shot inference often yields unreliable results for complex reasoning tasks, leading researchers to explore multiple reasoning paths through methods such as perplexity and self-consistency. In this paper, we present the first theoretical error decomposition analysis of these techniques, breaking down their error into estimation error and model error. Our analysis reveals a fundamental trade-off: perplexity methods suffer from substantial model error due to the absence of a proper consistency function, while self-consistency exhibits high estimation error due to a slow error convergence rate. To overcome these limitations, we propose Reasoning-Pruning Perplexity Consistency (RPC). This approach combines Perplexity Consistency, which seamlessly integrates LLM perplexity with self-consistency, and Reasoning Pruning, which eliminates low-probability reasoning paths to effectively prevent the degeneration of estimation error reduction. Theoretical analysis demonstrates that RPC not only accelerates the convergence rate of estimation error to an exponential level but also holds strong potential for further reducing model error. Extensive empirical evaluations on seven benchmark datasets confirm that RPC can significantly improve reasoning performance, sample efficiency, and confidence reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。