用动态图网络捕捉扩散模型生成过程中的幻觉线索
TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs
- 构建随时间演化的令牌级注意力图，追踪生成轨迹中的事实证据
- 在LLaDA-8B和Dream-7B上实现比基线更高的幻觉检测准确率
- 适合关注生成模型可信度与推理过程分析的研究者
扩散语言模型（D-LLMs）具备并行去噪与双向上下文能力，但其幻觉检测仍缺乏研究。传统针对自回归模型的检测方法依赖单次遍历线索，无法直接迁移至扩散生成场景——事实证据在去噪轨迹中分布、漂移或自我修正。本文提出TDGNet，一种基于时序动态图的幻觉检测框架，将检测问题建模为对演化令牌级注意力图的学习。每一步去噪中，通过消息传递稀疏化注意力图并更新各令牌记忆，再使用时序注意力聚合全轨迹证据进行最终预测。在LLaDA-8B和Dream-7B上的实验表明，该方法在问答基准上显著优于输出型、隐状态型及静态图基线，实现单次推理与低开销。结果凸显了在注意力图中进行时序推理对提升扩散语言模型幻觉检测鲁棒性的重要性。
原文摘要 · Abstract (English)
Diffusion language models (D-LLMs) offer parallel denoising and bidirectional context, but hallucination detection for D-LLMs remains underexplored. Prior detectors developed for auto-regressive LLMs typically rely on single-pass cues and do not directly transfer to diffusion generation, where factuality evidence is distributed across the denoising trajectory and may appear, drift, or be self-corrected over time. We introduce TDGNet, a temporal dynamic graph framework that formulates hallucination detection as learning over evolving token-level attention graphs. At each denoising step, we sparsify the attention graph and update per-token memories via message passing, then apply temporal attention to aggregate trajectory-wide evidence for final prediction. Experiments on LLaDA-8B and Dream-7B across QA benchmarks show consistent AUROC improvements over output-based, latent-based, and static-graph baselines, with single-pass inference and modest overhead. These results highlight the importance of temporal reasoning on attention graphs for robust hallucination detection in diffusion language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。