发现图神经网络中注意力机制会引发关键的极端激活,可用来解释分子键的预测逻辑。
Massive Activations in Graph Neural Networks: Decoding Attention for Domain-Dependent Interpretability
- 提出检测边特征注意力层中极端激活的新方法
- 极端激活集中在常见键类型上,避开高信息量三键
- 可作为自然归因指标,帮助理解模型决策依据
图神经网络(GNN)在建模图结构数据方面日益流行,注意力机制对捕捉复杂模式至关重要。本研究揭示了将注意力引入边特征GNN的一个关键但未被充分探讨的后果:注意力层中出现大规模激活(MAs)。通过开发一种针对边特征检测MAs的新方法,我们证明这些极端激活不仅是异常,还编码了与领域相关的信号。后处理可解释性分析显示,在分子图中,MAs主要聚集在常见键类型(如单键和双键),而避开更具信息量的三键。消融实验进一步证实,MAs可作为自然归因指标,重新分配至低信息量边。我们在基准数据集ZINC、TOX21和PROTEINS上评估了多种基于注意力的边特征GNN模型。主要贡献包括:(1) 建立注意力机制与边特征GNN中MAs生成之间的直接联系;(2) 提出稳健的MAs定义与检测方法,支持可靠的后处理可解释性。整体而言,本研究揭示了注意力机制、边特征GNN与MAs涌现之间的复杂相互作用,为连接GNN内部机制与领域知识提供了关键洞见。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have become increasingly popular for effectively modeling graph-structured data, and attention mechanisms have been pivotal in enabling these models to capture complex patterns. In our study, we reveal a critical yet underexplored consequence of integrating attention into edge-featured GNNs: the emergence of Massive Activations (MAs) within attention layers. By developing a novel method for detecting MAs on edge features, we show that these extreme activations are not only activation anomalies but encode domain-relevant signals. Our post-hoc interpretability analysis demonstrates that, in molecular graphs, MAs aggregate predominantly on common bond types (e.g., single and double bonds) while sparing more informative ones (e.g., triple bonds). Furthermore, our ablation studies confirm that MAs can serve as natural attribution indicators, reallocating to less informative edges. Our study assesses various edge-featured attention-based GNN models using benchmark datasets, including ZINC, TOX21, and PROTEINS. Key contributions include (1) establishing the direct link between attention mechanisms and MAs generation in edge-featured GNNs, (2) developing a robust definition and detection method for MAs enabling reliable post-hoc interpretability. Overall, our study reveals the complex interplay between attention mechanisms, edge-featured GNNs model, and MAs emergence, providing crucial insights for relating GNNs internals to domain knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。