提出DPA方法，高效追踪SwiGLU-Transformer的信息路径。
Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation
- 通过前向与反向传播，解析模型中信息流动路径。
- 在标准基准上实现最高忠实度与前所未有效率。
- 适合需要快速解释长序列模型的开发者使用。
理解基于Transformer的大语言模型内部机制对可靠部署和有效运行至关重要。尽管已有诸多归因方法试图在忠实度与计算效率间取得平衡，但密集组件归因仍成本高昂。本文提出双路径归因（Dual Path Attribution, DPA），一种新框架，可在一次前向和一次反向传播中，无需反事实样本，准确追踪冻结Transformer中的信息流。DPA将SwiGLU Transformer的计算结构解析并线性化为独立路径，沿路径传播目标解嵌向量，以获得每个残差位置的有效表示。该目标中心化传播实现相对于模型组件数量的O(1)时间复杂度，支持长输入序列与密集组件归因。在标准可解释性基准上的大量实验表明，相较于现有基线，DPA在忠实度上达到最先进水平，并实现了前所未有的效率。
原文摘要 · Abstract (English)
Understanding the internal mechanisms of transformer-based large language models (LLMs) is crucial for their reliable deployment and effective operation. While recent efforts have yielded a plethora of attribution methods attempting to balance faithfulness and computational efficiency, dense component attribution remains prohibitively expensive. In this work, we introduce Dual Path Attribution (DPA), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass without requiring counterfactual examples. DPA analytically decomposes and linearizes the computational structure of the SwiGLU Transformers into distinct pathways along which it propagates a targeted unembedding vector to receive the effective representation at each residual position. This target-centric propagation achieves O(1) time complexity with respect to the number of model components, scaling to long input sequences and dense component attribution. Extensive experiments on standard interpretability benchmarks demonstrate that DPA achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。