arXiv:2510.24299cs.AI2025-10被引 2

用相关矩阵秩判断大模型推理对错,无需额外训练。

Verifying Large Language Models' Reasoning Paths via Correlation Matrix Rank

  • 通过输入与输出路径的相关矩阵秩判断推理正确性。
  • 能准确区分对错推理路径,正确率超75%。
  • 方法简单高效,适合各类大模型通用验证。

尽管大语言模型具备强大的推理能力,但仍易出错和产生幻觉。现有验证方法多依赖外部资源,如训练好的验证器或复杂提示,计算开销大且仅适用于特定领域。本文探究大模型内部行为是否已蕴含推理可信度信息。发现输入问题与输出推理路径之间的相关矩阵秩是推理正确性的稳健指标。该方法仅依赖模型自身,无需额外训练或设计复杂提示。基于此,我们提出可即插即用的Self-Indicator方法,重新加权候选推理路径,在极低计算开销下显著优于其他投票与验证方法。跨多种规模和架构的大模型实验表明,Self-Indicator在三个推理基准上均提升超8%,并能以超过75%的准确率区分正确与错误推理路径。

原文摘要 · Abstract (English)

Despite the strong reasoning ability of large language models~(LLMs), they are prone to errors and hallucinations. As a result, how to check their outputs effectively and efficiently has become a critical problem in their applications. Existing checking methods heavily rely on external resources, such as trained verifiers (e.g., process/outcome reward models) or elaborate prompts, which lead to high computational overhead and are only applicable to specific domains. In this paper, we investigate whether the internal behaviors of LLMs have already implied the credibility of their reasoning paths. Specifically, we find that the rank of the correlation matrix between the input problem and the output reasoning path is a robust indicator of reasoning correctness. Different from other correctness indicators for LLMs, the calculation of the correlation matrix only relies on the LLM itself, which avoids the hassle of training a separate model or designing complicated prompts. Based on it, we design a simple, plug-and-play Self-Indicator method to reweight candidate reasoning paths, which achieves significant performance improvements than other voting and verification methods with very few computational overhead. Our experiments across multiple LLMs of varying scales and model families have further shown the effectiveness of Self-Indicator. It achieves over 75% accuracy in distinguishing correct reasoning paths from incorrect ones, and, in turn, improves the accuracies on three reasoning benchmarks by more than 8%.

大模型验证推理路径相关矩阵

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。