提出GAF方法，用最大流优化Transformer特征归因效果。
Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flow
- 将注意力权重与梯度结合，通过最大流求解特征重要性
- 在序列分类任务中显著优于现有归因方法
- 适合需要可解释性的Transformer模型研究者
本文提出广义注意力流（Generalized Attention Flow, GAF），一种针对基于Transformer模型的新型特征归因方法，旨在解决现有方法的局限性。通过扩展注意力流框架，以广义信息张量替代注意力权重，并融合注意力权重、其梯度、最大流问题及障碍法，提升特征归因性能。所提方法具备关键理论性质，克服了仅依赖注意力权重简单聚合的不足。在序列分类任务上的全面基准测试表明，GAF的一个特定变体在多数评估场景下持续优于当前最优特征归因方法，为Transformer模型输出提供了更可靠的可解释性。
原文摘要 · Abstract (English)
This paper introduces Generalized Attention Flow (GAF), a novel feature attribution method for Transformer-based models to address the limitations of current approaches. By extending Attention Flow and replacing attention weights with the generalized Information Tensor, GAF integrates attention weights, their gradients, the maximum flow problem, and the barrier method to enhance the performance of feature attributions. The proposed method exhibits key theoretical properties and mitigates the shortcomings of prior techniques that rely solely on simple aggregation of attention weights. Our comprehensive benchmarking on sequence classification tasks demonstrates that a specific variant of GAF consistently outperforms state-of-the-art feature attribution methods in most evaluation settings, providing a more reliable interpretation of Transformer model outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。