arXiv:2604.01624cs.AIcs.CL2026-04

用扩散模型的去噪轨迹实时检测幻觉,不依赖外部分类器。

OSCAR: Orchestrated Self-verification and Cross-path Refinement

  • 通过并行去噪链计算交叉熵,定位生成中不确定性高的词元位置。
  • 在TriviaQA等数据集上显著减少幻觉,提升事实准确率,尤其增强检索证据利用。
  • 无需训练,直接利用模型自身信号,适用于扩散语言模型部署场景。

扩散语言模型(DLMs)暴露其去噪轨迹,为推理阶段控制提供了天然接口;理想的幻觉缓解框架应利用模型原生信号进行干预,而非依赖外部训练的幻觉分类器。为此,我们提出‘承诺不确定性定位’:给定一个去噪轨迹,识别在事实不可靠的承诺传播前,跨链熵超过无监督阈值的词元位置。我们引入一系列轨迹级评估方法,包括跨链幻觉偏离度(CDH)指标,以系统比较定位方法。同时提出OSCAR,一种无需训练的推理时框架,实现该理念。OSCAR运行N个并行去噪链(N ∈ {4,8,16}),随机化揭示顺序,计算跨链香农熵以检测高不确定性位置,并基于检索证据进行定向重掩码。消融实验证明,定位与修正贡献互补增益,且对N鲁棒。在TriviaQA、HotpotQA、RAGTruth和CommonsenseQA上,使用LLaDA-8B和Dream-7B,OSCAR通过不确定性引导的重掩码显著降低幻觉内容,提升事实准确性,并更有效整合检索证据。其基于熵的原生不确定性信号优于专用训练探测器,凸显扩散语言模型固有的事实不确定性识别能力,此能力在自回归模型的序列承诺结构中不存在。

原文摘要 · Abstract (English)

Diffusion language models (DLMs) expose their denoising trajectories, offering a natural handle for inference-time control; accordingly, an ideal hallucination mitigation framework should intervene during generation using this model-native signal rather than relying on an externally trained hallucination classifier. Toward this, we formulate commitment uncertainty localization: given a denoising trajectory, identify token positions whose cross-chain entropy exceeds an unsupervised threshold before factually unreliable commitments propagate into self-consistent but incorrect outputs. We introduce a suite of trajectory-level assessments, including a cross-chain divergence-at-hallucination (CDH) metric, for principled comparison of localization methods. We also introduce OSCAR, a training-free inference-time framework operationalizing this formulation. OSCAR runs N parallel denoising chains with randomized reveal orders, computes cross-chain Shannon entropy to detect high-uncertainty positions, and then performs targeted remasking conditioned on retrieved evidence. Ablations confirm that localization and correction contribute complementary gains, robust across N in {4, 8, 16}. On TriviaQA, HotpotQA, RAGTruth, and CommonsenseQA using LLaDA-8B and Dream-7B, OSCAR enhances generation quality by significantly reducing hallucinated content and improving factual accuracy through uncertainty-guided remasking, which also facilitates more effective integration of retrieved evidence. Its native entropy-based uncertainty signal surpasses that of specialized trained detectors, highlighting an inherent capacity of diffusion language models to identify factual uncertainty that is not present in the sequential token commitment structure of autoregressive models.

扩散模型幻觉检测推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。