提出新方法提升Transformer文本分类的可解释性，让模型决策更可信。
Contrast-CAT: Contrasting Activations for Enhanced Interpretability in Transformer-based Text Classifiers
- 通过对比输入与参考激活，过滤无关特征，优化注意力分配。
- 在多个数据集上显著优于现有方法，平均提升AOPC 1.30倍、LOdds 2.25倍。
- 适合需要高可信度解释的NLP应用，如医疗、金融等安全敏感场景。
Transformer深刻影响了人工智能研究，但其决策解释仍具挑战性，尤其在分类任务中，这阻碍了其在真实场景中的信任与安全部署。尽管基于激活的归因方法能有效解释Transformer文本分类模型，但我们发现这些方法易受激活中与类别无关特征的干扰，导致解释不可靠。为此，我们提出Contrast-CAT，一种基于激活对比的新型归因方法，通过对比输入序列与参考激活，过滤类无关特征，生成更清晰、更忠实的逐标记归因图。在多个数据集和模型上的实验结果表明，Contrast-CAT持续优于当前最优方法。值得注意的是，在MoRF设置下，其在AOPC上平均提升1.30倍，在LOdds上平均提升2.25倍，充分证明了该方法在增强Transformer文本分类可解释性方面的有效性。
原文摘要 · Abstract (English)
Transformers have profoundly influenced AI research, but explaining their decisions remains challenging -- even for relatively simpler tasks such as classification -- which hinders trust and safe deployment in real-world applications. Although activation-based attribution methods effectively explain transformer-based text classification models, our findings reveal that these methods can be undermined by class-irrelevant features within activations, leading to less reliable interpretations. To address this limitation, we propose Contrast-CAT, a novel activation contrast-based attribution method that refines token-level attributions by filtering out class-irrelevant features. By contrasting the activations of an input sequence with reference activations, Contrast-CAT generates clearer and more faithful attribution maps. Experimental results across various datasets and models confirm that Contrast-CAT consistently outperforms state-of-the-art methods. Notably, under the MoRF setting, it achieves average improvements of x1.30 in AOPC and x2.25 in LOdds over the most competing methods, demonstrating its effectiveness in enhancing interpretability for transformer-based text classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。