提出LIG方法,解析Transformer层内信息流动路径。
LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers

- 在层内模块边界使用集到集的积分梯度,量化输入输出token间贡献
- 发现注意力基线用目标词嵌入、MLP基线用零或ATT输出时一致性最佳
- 无需重训练即可诊断模型内部机制,适合可解释性研究者
Transformer性能强大但内部计算不透明。本文将每一层视为动态图,节点为标记表示和各头注意力输出,以多头注意力(ATT)和MLP作为模块边界。在该图上应用分层积分梯度(LIG),即在非线性模块边界使用集到集的积分梯度(IG)。集到集IG将输入标记表示集合映射到输出表示集合,评估标记间的贡献关系,不同于以往标准的标量目标设置。通过L2标量化将IG扩展至集到集映射,并借鉴层内传播思想(类似LRP),利用积分梯度完备性实现每边界的守恒特性。使用LIG分析:(i) 模块组合与整层归因在L2准则下的一致性;(ii) 层内信息流,追踪分离的注意力与MLP贡献。在BERT-base和PTB数据集上,最佳保持层内一致性的配置是:注意力基线采用目标词嵌入,MLP基线采用a=0时的注意力输出或零。因此,本文将LIG呈现为一种无需模型特定重训练或操作级解释器设计的模块边界粒度诊断XAI工具。代码已公开于https://github.com/eightsuzuki/layer-wise-integrated-gradients。
原文摘要 · Abstract (English)
Transformers achieve strong performance, but their internal computations remain opaque. We view each Transformer layer as a dynamic graph whose nodes are token representations and per-head attention outputs, with Multi-Head Attention (ATT) and MLP as module boundaries. On this graph we use LIG (Layer-wise Integrated Gradients), which applies set-to-set Integrated Gradients (IG) at nonlinear module boundaries. Set-to-set IG applies IG to a map from a set of input token representations to a set of output representations, evaluating token-to-token contributions, which is not standard in prior IG applications. This extends IG from the usual scalar-objective setting to set-to-set maps via an L2 scalarization, and composes within-layer contributions in the spirit of Layer-wise Relevance Propagation (LRP), with IG completeness playing the role of LRP-style conservation at each boundary. We use LIG to analyze (i) the agreement between module-wise composition and layer-whole attribution under an L2 criterion, and (ii) within-layer information flow by tracing separated ATT and MLP contributions. On BERT-base and PTB, configurations that best preserved within-layer consistency used the target token's embedding as the ATT baseline and either the ATT output at a=0 or Zero as the MLP baseline. We therefore present LIG as a diagnostic XAI tool at module-boundary granularity, without model-specific retraining or per-operation interpreter design. Code is available at https://github.com/eightsuzuki/layer-wise-integrated-gradients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。