DePass通过分解前向传播，精准追踪Transformer模型内部信息流动。
DePass: Unified Feature Attributing by Simple Decomposed Forward Pass
- 单次分解前向传播，拆解隐藏状态为可追踪成分
- 无需额外训练，实现细粒度且可信的特征归因
- 适用于任意组件间的信息流分析，适合模型可解释性研究
揭示Transformer模型内部计算行为是机制可解释性的核心挑战。我们提出DePass，一种基于单次分解前向传播的统一特征归因框架。DePass将隐藏状态分解为自定义的加性成分，并在注意力分数和MLP激活值固定的情况下进行传播。该方法无需辅助训练即可实现忠实且细粒度的归因，在词元级、模型组件级和子空间级任务中均验证了其有效性与保真度。实验表明，DePass具备对Transformer模型任意组件间信息流进行归因的潜力。我们期望它能成为可解释性研究中的基础工具。
原文摘要 · Abstract (English)
Attributing the behavior of Transformer models to internal computations is a central challenge in mechanistic interpretability. We introduce DePass, a unified framework for feature attribution based on a single decomposed forward pass. DePass decomposes hidden states into customized additive components, then propagates them with attention scores and MLP's activations fixed. It achieves faithful, fine-grained attribution without requiring auxiliary training. We validate DePass across token-level, model component-level, and subspace-level attribution tasks, demonstrating its effectiveness and fidelity. Our experiments highlight its potential to attribute information flow between arbitrary components of a Transformer model. We hope DePass serves as a foundational tool for broader applications in interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。