提出新方法让医学与航拍图像的热力图旋转不变，解释更可信。
Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging
- 通过多视角逆旋转平均，实现无需训练的热力图旋转对齐。
- 在ImageNet上提升36%-247%的旋转等变性，优于数据增强训练。
- 可识别并剔除被分类器丢弃的无关自由度，解释更稳定可靠。
后处理显著性图如Grad-CAM被广泛用于解释视觉模型决策，但在输入旋转时即使预测不变，热力图也会漂移。在无标准朝向的领域（如病理学与航拍图像），这削弱了显著性作为证据的可信度。本文通过测量整个算子各阶段的等变性，而非仅从输出推断，来判断该漂移是真实信号还是由算子引入的噪声。发现最稳定的阶段是通道权重，且在ResNet-50中完全稳定，因全局平均池化+线性头使类别梯度场空间恒定；真正变化的是空间激活张量，而分类器池化操作会丢弃这种变化。因果测试显示：遮挡显著性漂移区域比遮挡随机像素造成的损失更小。漂移来自分类器丢弃的自由度，因此消除它并非破坏性操作。EquiGrad-CAM是一种无需训练的封装方法，对T个旋转视图分别逆旋转其显著性至统一参考系后求平均。在ImageNet-1K全集上，相比单视图Grad-CAM，其等变性分别提升+36.0%（ResNet-50）、+87.5%（VGG-16）和+247%（ViT-B/16）；缩放匹配消融实验表明，对齐而非聚合是性能提升的关键。该方法优于旋转增强训练，不需重训练即可提升零样本CLIP解释一致性+145%，并在PatchCamelyon与RESISC45上获得旋转一致的解释。附带产物PEUM可免费评估解释可复现性，仅需已有视图。
原文摘要 · Abstract (English)
Post-hoc saliency maps such as Grad-CAM are increasingly used to audit why a deployed vision model made a decision, yet the heatmap drifts when the input is rotated, even when the prediction is unchanged. In domains with no canonical orientation, such as histopathology and aerial imagery, this undermines using saliency as evidence. We ask whether that drift is faithful signal or noise introduced by the CAM operator, and answer it by measuring equivariance at every stage of the operator rather than inferring it from the network's output. The instability is not where one would guess: the channel weights are the most rotation-stable stage, and on ResNet-50 exactly stable, because a GAP+linear head makes the class gradient field spatially constant. What moves is the spatial activation tensor, and the classifier's own pooling discards that movement. A causal test confirms the consequence: occluding the pixels whose saliency drifts costs the model less than occluding random pixels, at either orientation. The drift is carried by degrees of freedom the classifier throws away, which is what makes removing it faithful rather than destructive. EquiGrad-CAM is a training-free wrapper that takes T rotated views, inverse-rotates each view's saliency into a common canonical frame, and averages. On the full ImageNet-1K validation set it raises equivariance over single-view Grad-CAM by +36.0% (ResNet-50), +87.5% (VGG-16) and +247% (ViT-B/16); a scale-matched ablation isolates alignment before averaging, not the locus of aggregation, as the driver. It beats rotation-augmented training without retraining, lifts zero-shot CLIP by +145%, and yields rotation-consistent explanations on PatchCamelyon and RESISC45. Its by-product PEUM ranks explanations by how reproducible they are, at no cost beyond the views already taken. Code: https://github.com/Khawaja-Murad/EquiGrad-CAM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。