arXiv:2507.21723cs.CVcs.AI2025-07

通过神经科学启发的消融实验,揭示检测Transformer内部组件的作用与冗余。

Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations

  • 借鉴脑区损毁研究,系统切除检测Transformer的关键组件以分析其功能。
  • 发现DETR对编码器和解码器注意力层最敏感,而DINO因双次前向更新最鲁棒。
  • 揭示了模型结构冗余,为简化网络提升效率提供依据,适合模型优化研究者。

近年来,可解释人工智能在提升复杂模型如检测变压器的可解释性方面日益受到关注。尽管进展迅速,但对内部组件各自作用的理解仍存在显著空白,而这对于提升透明度和效率至关重要。受神经科学中通过选择性损伤研究脑区功能的启发,我们系统分析了三种前沿检测变压器模型——检测变压器(DETR)、可变形检测变压器(DDETR)和改进去噪锚框的DINO——中关键组件的消融影响,包括查询嵌入、编码器与解码器多头自注意力(MHSA)以及解码器多头交叉注意力(MHCA)层。我们在COCO数据集上评估了这些消融对gIoU和F1-score的影响,量化了其在分类与回归任务中的表现变化。为促进可复现性与未来研究,我们公开发布了DeepDissect库。结果表明,不同模型具有特定的鲁棒性模式:DETR对编码器MHSA和解码器MHCA消融尤为敏感,而DDETR因多尺度可变形注意力更具鲁棒性,DINO则因‘两次前向更新’机制分布知识,表现出最强韧性。这些发现还暴露了结构性冗余,尤其是在DDETR和DINO的解码器MHCA层,表明可在不牺牲性能的前提下进行模型简化。本研究推进了检测变压器的XAI研究,明确了内部组件对性能的贡献,为优化与提升关键应用中的透明度与效率提供了洞见。

原文摘要 · Abstract (English)

In recent years, Explainable AI has gained traction as an approach to enhancing model interpretability and transparency, particularly in complex models such as detection transformers. Despite rapid advancements, a substantial research gap remains in understanding the distinct roles of internal components - knowledge that is essential for improving transparency and efficiency. Inspired by neuroscientific ablation studies, which investigate the functions of brain regions through selective impairment, we systematically analyze the impact of ablating key components in three state-of-the-art detection transformer models: Detection transformer (DETR), deformable detection transformer (DDETR), and DETR with improved denoising anchor boxes (DINO). The ablations target query embeddings, encoder and decoder multi-head self-attentions (MHSA) as well as decoder multi-head cross-attention (MHCA) layers. We evaluate the effects of these ablations on the performance metrics gIoU and F1-score, quantifying effects on both the classification and regression sub-tasks on the COCO dataset. To facilitate reproducibility and future research, we publicly release the DeepDissect library. Our findings reveal model-specific resilience patterns: while DETR is particularly sensitive to ablations in encoder MHSA and decoder MHCA, DDETR's multi-scale deformable attention enhances robustness, and DINO exhibits the greatest resilience due to its look-forward twice update rule, which helps distributing knowledge across blocks. These insights also expose structural redundancies, particularly in DDETR's and DINO's decoder MHCA layers, highlighting opportunities for model simplification without sacrificing performance. This study advances XAI for DETRs by clarifying the contributions of internal components to model performance, offering insights to optimize and improve transparency and efficiency in critical applications.

可解释AI检测模型消融实验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。