让检测模型自动生成精准的解释图,效率比传统方法快得多。
EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers

- 用注意力机制直接生成每个物体的解释图,不需反向传播或扰动输入。
- 在COCO、ExDark等数据集上效果优于现有方法,且解释速度提升数倍。
- 适合需要高效可解释性的目标检测应用,如自动驾驶、医疗影像。
目标检测的视觉可解释性因多实例特性而困难。现有方法多采用后处理范式(如基于梯度或扰动的方法),需额外计算梯度或重复推理,效率受限。为此,本文提出端到端实例级视觉解释框架EIVE,直接在检测变压器(DETR)类模型前向传播后生成实例级显著图。具体地,将解码器中的交叉注意力机制重构为实例级特征归因路径,使每个对象查询的交叉注意力对应其预测实例的视觉归因。在此基础上,设计跨层混合共识融合(CLHCF)模块,聚合解码器各层的注意力信号,生成稳定紧凑的解释。EIVE无需梯度计算或输入扰动,计算效率高,适用于单尺度和多尺度DETR类检测器。最后提出一种注意力感知联合训练策略(AAJTS),对交叉注意力模式施加空间约束,促进归因表示的稳定集中,从而提升可解释性与检测性能。在MS COCO 2017、ExDark和Cityscapes上的实验表明,EIVE生成高质量实例级显著图,在标准指标上表现媲美或优于前沿后处理方法,同时显著提升解释效率。代码已开源。
原文摘要 · Abstract (English)
Visual explainability for object detection remains challenging due to the multi-instance nature of detection. Existing approaches predominantly adopt post-hoc paradigms, such as gradient-based or perturbation-based explanation methods, to interpret pretrained detectors. However, these methods require additional gradient computation or repeated model inference, resulting in limited efficiency. To address this issue, we propose an End-to-end Instance-specific Visual Explanation framework (EIVE) that directly generates instance-level saliency maps following the forward pass of Detection Transformer (DETR)-like models. Specifically, we reformulate the cross-attention mechanism in the decoder as an instance-level feature attribution pathway, so that the cross-attention of each object query corresponds to the visual attribution of its predicted instance. Based on this formulation, we design a cross-layer hybrid consensus fusion (CLHCF) module to aggregate cross-attention signals across decoder layers, producing stable and compact explanations. The explanation process of EIVE requires neither gradient computation nor input perturbation, yielding high computational efficiency, and applies to single- and multi-scale DETR-like object detectors. Finally, we present an attention-aware joint training strategy (AAJTS) as a training-oriented application, which imposes spatial constraints on cross-attention patterns to encourage stable and concentrated attribution representations, thereby improving both interpretability and detection performance. Experiments on MS COCO 2017, ExDark, and Cityscapes demonstrate that EIVE produces high-quality instance-level saliency maps and achieves performance comparable to, or better than, state-of-the-art post-hoc methods across standard metrics, while substantially improving explanation efficiency. Code is available at https://github.com/xjlDestiny/EIVE.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。