研究RNN模型中注意力机制是否能解释自然语言推理的决策逻辑
A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference
- 用双RNN编码器对比注意力权重与人类标注
- 启发式方法与人工标注相关性较高，可辅助评估解释合理性
- 原始注意力权重难以提供可信解释，需谨慎解读
自然语言处理模型中的注意力图常被用来解释模型决策，理想情况下应聚焦于支持判断的关键词汇。尽管已有大量实证研究通过典型案例表明注意力图具备解释潜力，但真正评估其对人类理解决策的合理性（即解释可信度）的研究仍较少，且多集中于文本分类任务。本文首次在句子比较任务——自然语言推理中开展初步评估，基于eSNLI数据集，将双RNN编码器间的交叉注意力权重与人工标注及启发式标注进行对比。结果表明，启发式标注与人工标注具有较好相关性，可用于评估句子比较任务中解释的合理性；而原始注意力权重与合理解释之间的关联较弱。
原文摘要 · Abstract (English)
Attention maps in neural models for NLP are appealing to explain the decision made by a model, hopefully emphasizing words that justify the decision. While many empirical studies hint that attention maps can provide such justification from the analysis of sound examples, only a few assess the plausibility of explanations based on attention maps, i.e., the usefulness of attention maps for humans to understand the decision. These studies furthermore focus on text classification. In this paper, we report on a preliminary assessment of attention maps in a sentence comparison task, namely natural language inference. We compare the cross-attention weights between two RNN encoders with human-based and heuristic-based annotations on the eSNLI corpus. We show that the heuristic reasonably correlates with human annotations and can thus facilitate evaluation of plausible explanations in sentence comparison tasks. Raw attention weights however remain only loosely related to a plausible explanation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。