arXiv:2603.16459cs.CL2026-03中稿 · EMNLP被引 10

通过分析去噪动态偏差,提升扩散型大模型幻觉检测效果。

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

  • 按语义重要性筛选关键token,避免冗余信息干扰
  • 建模不确定性随去噪过程的演变轨迹,识别异常偏差
  • 在多个模型和数据集上优于现有方法,且效率更高

扩散型大语言模型(D-LLMs)因其迭代优化能力成为自回归模型的有力替代,但幻觉问题仍严重影响其可靠性。现有方法常使用分词级不确定性(如熵值)来指示事实错误,但D-LLMs以固定长度并行生成文本,仅有少数关键词对幻觉检测有用,整体平均不确定性会降低效果。此外,不确定性在去噪过程中的演化趋势也蕴含重要信号,因此建模其去噪动态至关重要。本文提出DynHD,从空间(分词序列)与时间(去噪动态)双维度解决该问题。设计语义感知证据构建模块,剔除任务无关结构词,聚焦剩余关键词的不确定性;引入参考证据生成器学习预期的不确定性演化路径,并通过观测路径与参考路径的偏差进行幻觉判断。大量实验表明,DynHD在多个基准与主干模型上持续优于当前最优基线,且具有更高效率。代码已开源:https://github.com/qyy11-com/DynHD。

原文摘要 · Abstract (English)

Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty, such as entropy, has been widely used to indicate potential factual errors. Nevertheless, unlike auto-regressive models that generate tokens sequentially, D-LLMs generate fixed-length sequences simultaneously, where only a small subset of tokens is informative for hallucination detection. Thus, aggregating uncertainty over all tokens can be suboptimal. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide valuable signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD, which bridges these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To handle the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by removing task-invariant structural tokens and emphasizing the uncertainty of the remaining informative tokens. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models. The code is available at: https://github.com/qyy11-com/DynHD.

幻觉检测扩散模型不确定性建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。