arXiv:2504.08602cs.CVcs.AI2025-04被引 1

发现视觉模型概念嵌入会受背景干扰,导致对特定场景识别失效。

On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNs

  • 通过对比多种数据驱动方法,分析背景对概念嵌入的影响机制。
  • 在50多个概念、2个数据集上验证,多数方法在道路场景下性能显著下降。
  • 揭示了现有解释方法本身存在背景偏见,适合关注模型鲁棒性的研究者。

概念可解释人工智能(C-XAI)研究关注人类可理解的语义概念如何嵌入深度神经网络(DNN)的隐空间。后处理方法通过一组示例定义概念,并使用数据驱动技术确定其在DNN隐空间中的嵌入。这类方法已成功揭示不同目标类别间的偏差。然而,由于训练过程中背景大多未受控制,一个关键问题仍未解决:当前主流的数据驱动后处理C-XAI方法是否自身也受背景偏见影响?例如,野生动物通常出现在植被背景中,极少出现在道路上。即使简单稳健的C-XAI方法也可能利用这一捷径提升表现,导致动物在道路场景下的识别性能下降被掩盖。本工作验证并全面确认,基于Net2Vec的概念分割技术常捕捉到背景偏见,包括严重问题如道路场景下表现不佳。我们对比了3种背景随机化技术,在超过50个概念和2个数据集上,涵盖7种不同DNN架构。结果表明,低成本设置即可提供有价值洞察并提升背景鲁棒性。

原文摘要 · Abstract (English)

The thriving research field of concept-based explainable artificial intelligence (C-XAI) investigates how human-interpretable semantic concepts embed in the latent spaces of deep neural networks (DNNs). Post-hoc approaches therein use a set of examples to specify a concept, and determine its embeddings in DNN latent space using data driven techniques. This proved useful to uncover biases between different target (foreground or concept) classes. However, given that the background is mostly uncontrolled during training, an important question has been left unattended so far: Are/to what extent are state-of-the-art, data-driven post-hoc C-XAI approaches themselves prone to biases with respect to their backgrounds? E.g., wild animals mostly occur against vegetation backgrounds, and they seldom appear on roads. Even simple and robust C-XAI methods might abuse this shortcut for enhanced performance. A dangerous performance degradation of the concept-corner cases of animals on the road could thus remain undiscovered. This work validates and thoroughly confirms that established Net2Vec-based concept segmentation techniques frequently capture background biases, including alarming ones, such as underperformance on road scenes. For the analysis, we compare 3 established techniques from the domain of background randomization on >50 concepts from 2 datasets, and 7 diverse DNN architectures. Our results indicate that even low-cost setups can provide both valuable insight and improved background robustness.

可解释性背景偏见概念嵌入鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。