通过多视角注意力特征检测大模型生成中的幻觉片段
Hallucinated Span Detection with Multi-View Attention Features
- 从注意力矩阵提取三类互补特征：关键性、偏倚性与上下文范围
- 在长输入任务中优于现有方法，尤其在数据到文本和摘要任务上
- 适合关注生成质量评估与幻觉检测的研究者使用
本研究针对大语言模型输出中存在的幻觉片段检测问题，该问题虽具实际重要性但关注度低于输出层面的幻觉检测。已有研究表明，幻觉发生时注意力模式常呈现异常。受此启发，本文从注意力矩阵中提取三类互补特征：(a) 某些词元是否具有影响力或被忽略，(b) 注意力是否偏向特定子集，(c) 生成时参考的是窄还是广的上下文。这些特征输入基于Transformer的分类器，进行序列标注以识别幻觉片段。实验结果表明，该方法在长输入场景（如数据到文本和摘要任务）中优于多个强基线模型。
原文摘要 · Abstract (English)
This study addresses the problem of hallucinated span detection in the outputs of large language models. It has received less attention than output-level hallucination detection despite its practical importance. Prior work has shown that attentions often exhibit irregular patterns when hallucinations occur. Motivated by these findings, we extract features from the attention matrix that provide complementary views capturing (a) whether certain tokens are influential or ignored, (b) whether attention is biased toward specific subsets, and (c) whether a token is generated referring to a narrow or broad context, in the generation. These features are input to a Transformer-based classifier to conduct sequential labelling to identify hallucinated spans. Experimental results indicate that the proposed method outperforms strong baselines on hallucinated span detection with longer input contexts, such as data-to-text and summarisation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。