arXiv:2411.02833cs.CV2024-11被引 1

研究上下文对图像识别中特征归因的影响,揭示模型对上下文依赖的深层机制。

Lost in Context: The Influence of Context on Feature Attribution Methods for Object Recognition

  • 用多种归因方法分析模型在不同上下文下的特征依赖性。
  • 正确分类时模型更关注物体本身而非上下文特征。
  • 上下文变化比扰动对模型性能影响更大,且无信息场景下上下文仍具意义。

上下文信息在计算机视觉中的物体识别模型中起着关键作用,上下文变化会显著影响模型准确率,凸显模型对上下文线索的依赖。本研究探究上下文操纵如何影响模型准确率与特征归因,通过特征归因方法揭示深度神经网络在物体识别任务中对上下文的依赖程度。我们采用多种特征归因技术,在ImageNet-9和自建的ImageNet-CS数据集上进行实验,分析上下文变化的影响。研究发现:(a) 正确分类的图像主要强调物体体积的归因,而非上下文体积;(b) 不同上下文修改下,模型对上下文的依赖保持相对稳定,不受分类准确率影响;(c) 上下文改变对模型性能的影响大于上下文扰动;(d) 令人意外的是,在‘无信息’场景中,上下文归因仍具有非平凡意义。本研究突破传统方法,评估对象或其上下文发生大规模变化时对物体识别的影响。

原文摘要 · Abstract (English)

Contextual information plays a critical role in object recognition models within computer vision, where changes in context can significantly affect accuracy, underscoring models' dependence on contextual cues. This study investigates how context manipulation influences both model accuracy and feature attribution, providing insights into the reliance of object recognition models on contextual information as understood through the lens of feature attribution methods. We employ a range of feature attribution techniques to decipher the reliance of deep neural networks on context in object recognition tasks. Using the ImageNet-9 and our curated ImageNet-CS datasets, we conduct experiments to evaluate the impact of contextual variations, analyzed through feature attribution methods. Our findings reveal several key insights: (a) Correctly classified images predominantly emphasize object volume attribution over context volume attribution. (b) The dependence on context remains relatively stable across different context modifications, irrespective of classification accuracy. (c) Context change exerts a more pronounced effect on model performance than Context perturbations. (d) Surprisingly, context attribution in `no-information' scenarios is non-trivial. Our research moves beyond traditional methods by assessing the implications of broad-level modifications on object recognition, either in the object or its context.

特征归因上下文依赖图像识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。