提出注意力头影响得分，量化其对分类决策的实际贡献。
Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time
- 融合方向性影响与残差流结构贡献，多尺度评估注意力头
- 在DeBERTa模型上区分正确与错误预测的决策行为
- 适合研究Transformer分类器内部决策机制的研究者
我们提出一种影响得分，用于量化基于Transformer的模型中注意力头在提示注入检测任务中对分类决策的贡献。该得分结合了对输出logits的方向性影响与在残差流中的结构贡献，支持在注意力头、层及网络层级进行多尺度分析。应用于专用于提示注入检测的DeBERTa模型时，该框架揭示了正确与错误预测之间不同的决策行为。该方法在细粒度电路分析与全局输出方法之间取得有效平衡，为研究Transformer分类器的决策机制提供了系统性途径。
原文摘要 · Abstract (English)
We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence on the logits with structural contribution within the residual stream, enabling a multi-scale analysis at the head, layer, and network levels. Applied to a DeBERTa model specialized for prompt injection detection, our framework reveals distinct decision behaviours between correct and erroneous predictions. Our method provides an effective compromise between fine-grained circuit analysis and global output-based methods, and offers a systematic way to study decision mechanisms in Transformer classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。