给大模型的推理过程去噪,提升幻觉检测准确率
Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models

- 用最终答案注意力作为自动监督信号,优化每一步推理表示
- 在多个基准上显著提升幻觉检测效果,最高提升12.3个百分点
- 可无缝接入各类检测器,适合需要可信推理的AI系统
大型推理模型在生成最终答案前会输出较长的推理轨迹。尽管这些轨迹可能包含有助于幻觉检测的线索,但长轨迹中常含噪声步骤,干扰真伪判断。本文识别出两类常见噪声:无关步骤和重复步骤,二者均显著降低检测性能。现有基于置信度的评分与简单嵌入过滤方法难以有效区分噪声与有用信息。为此,我们提出REDE框架,利用最终答案注意力作为自动监督信号,构建更优的步骤级表示空间,从而可靠识别并过滤噪声步骤。该方法可直接应用于多种幻觉检测器,仅需对去噪后的推理轨迹进行处理。在多个推理基准上的实验表明,REDE持续优于现有基线,平均提升达12.3%。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) generate long reasoning traces before producing final answers. While these traces may contain useful signals for hallucination detection, harnessing them is non-trivial because long trajectories often include noisy steps that obscure the cues relevant to truthfulness assessment. In this paper, we identify two prevalent forms of reasoning noises, i.e., irrelevant steps and repetitive steps, and show that both substantially degrade hallucination detection performance. Existing confidence-based scores and naive embedding-based filtering fail to reliably separate noisy from informative steps. To address this challenge, we propose REDE, a novel learning framework for denoising reasoning traces for hallucination detection. Specifically, REDE leverages final-answer attention as an automatic supervision signal to shape the step-level representation space, yielding refined embeddings in which noisy steps can be reliably identified and filtered. REDE can be readily plugged into diverse hallucination detectors by operating on the filtered reasoning trajectory after removing noisy steps. Extensive experiments on multiple reasoning benchmarks show that REDE consistently improves detection performance over competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。