arXiv:2606.29693cs.LG2026-06

精确计算Transformer各层对预测概率的贡献,解决传统方法的偏差与不连续问题。

IG-Lens: Exact Additive Probability Attribution Across Transformer Layers via Telescoping Integrated Gradients

  • 通过望远镜式积分梯度,将概率变化按层精确分解。
  • 结果在浮点精度下完全满足可加性,无离散化误差。
  • 单次前向传播即可完成全层贡献计算,适合高效分析模型决策过程。

我们提出IG-Lens,一种基于望远镜式积分梯度的精确概率归因方法,用于解码器仅用的Transformer模型。现有层级读出工具仅能近似估计每层的概率贡献,且存在偏差或非单调问题。直接梯度法虽可加但仅在logit空间有效,而软最大值非线性破坏了概率空间的可加性。层导通法虽逐层积分,但以各自基线为参考,无法总和到整体变化。IG-Lens沿隐藏状态路径从基线到最终层应用积分梯度,将每个段落归因于其终止层,使总和恰好等于目标概率的变化量,且软最大值嵌入积分路径中而非被线性化。默认估计器采用预测感知重加权,以每步的实际概率变化代替原始梯度。由于读出是一维概率,每段可简化为端点值的望远镜求和,从而在任意步数下精确满足完备性(至浮点精度),消除黎曼离散误差并抑制无输出变化却有梯度敏感的步骤。论文给出望远镜恒等式及其证明,验证了浮点精度下的完备性,并提供无需反向传播的单次前向批处理实现。代码已开源:https://github.com/anhnda/IGLens。

原文摘要 · Abstract (English)

We ask a simple question about decoder-only transformers: between which two layers is the probability of a predicted token actually produced? Existing layer-wise readout tools answer only approximately. The logit lens and its trained variant report a per-layer level of probability but give no additive decomposition; their estimates are biased and non-monotone across depth. Direct Logit Attribution and related residual-stream methods are additive, but only in logit space, the softmax nonlinearity breaks additivity in probability space, precisely the quantity one usually cares about. Layer Conductance integrates gradients per layer, but attributes each to its own baseline and so does not sum to the total change in prediction. We introduce IG-Lens, a telescoping application of Integrated Gradients along a single path through the hidden states from a baseline to the final layer. Crediting each segment to the layer it terminates at yields a layer-wise attribution whose sum is exactly the change in target probability, with the softmax inside the integration path rather than linearized away. Our default estimator credits each integration step its observed change in target probability (a prediction-aware reweighting in the spirit of IDGI) rather than its raw gradient. Because the readout is a one-dimensional probability, this collapses each segment to a telescoping sum of endpoint values, so completeness holds exactly (to floating point) at any step count, removing Riemann discretization error while suppressing steps that show gradient sensitivity without a change in output. We give the telescoping identity and its proof, verify completeness to floating point, and describe a single-pass batched implementation computing the full token-by-layer map without any backward call. Code: https://github.com/anhnda/IGLens.

可解释性Transformer归因分析积分梯度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。