提出双向注意力方法，让ViT模型解释更全面且能发现对抗样本。
What Helps---and What Hurts: Bidirectional Explanations for Vision Transformers
- 通过保留正负双向贡献，生成更完整的视觉解释
- 在多个数据集上提升定位精度与解释可信度
- 无需重训练即可检测对抗样本，适合安全敏感场景
视觉变压器（ViTs）在图像识别中表现优异，但其决策过程难以解释。本文提出双向类激活映射方法BiCAM，同时捕捉支持性（正向）和抑制性（负向）的贡献。与以往忽略负向信号的CAM方法不同，BiCAM保留带符号的归因，生成更完整、更具对比性的解释。此外，引入正向-负向比（PNR）量化归因平衡，可无需重新训练即实现对抗样本的轻量级检测。在ImageNet、VOC和COCO等多个数据集上，BiCAM均提升了定位准确性和解释忠实度，并兼容DeiT、Swin等多种ViT结构。结果表明，在解释基于变压器的视觉模型时，同时建模支持性和抑制性证据至关重要。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) achieve strong performance in visual recognition, yet their decision-making remains difficult to interpret. We propose BiCAM, a bidirectional class activation mapping method that captures both supportive (positive) and suppressive (negative) contributions to model predictions. Unlike prior CAM-based approaches that discard negative signals, BiCAM preserves signed attributions to produce more complete and contrastive explanations. BiCAM further introduces a Positive-to-Negative Ratio (PNR) that summarizes attribution balance and enables lightweight detection of adversarial examples without retraining. Across ImageNet, VOC, and COCO, BiCAM improves localization and faithfulness while remaining computationally efficient. It generalizes to multiple ViT variants, including DeiT and Swin. These results suggest the importance of modeling both supportive and suppressive evidence for interpreting transformer-based vision models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。