研究图像中物体位置和大小如何影响模型对背景的依赖。
Corner Cases: How Size and Position of Objects Challenge ImageNet-Trained Models
- 构建合成数据集 Hard-Spurious-ImageNet，控制物体位置与大小变化。
- 小目标且偏角落时，模型严重依赖背景特征，准确率下降明显。
- 现有去偏方法忽略位置大小影响，对极端情况效果不佳。
图像背景在数据集中常引发虚假相关性。由于人类拍摄习惯，不同类别存在位置（物体在画面中的位置）和尺寸（感兴趣区域与图像比例）偏差。本文通过基于 ImageNet-1k 构建的合成数据集 Hard-Spurious-ImageNet，揭示当目标区域与图像比例小且远离中心时，多数预训练模型会过度依赖背景特征进行预测。实验表明，当前缓解虚假特征的方法未考虑位置与尺寸因素，在最差组准确率上未能取得显著提升。代码与数据集已公开于 https://github.com/Mishalfatima/Corner_Cases。
原文摘要 · Abstract (English)
Backgrounds in images play a major role in contributing to spurious correlations among different data points. Owing to aesthetic preferences of humans capturing the images, datasets can exhibit positional (location of the object within a given frame) and size (region-of-interest to image ratio) biases for different classes. In this paper, we show that these biases can impact how much a model relies on spurious features in the background to make its predictions. To better illustrate our findings, we propose a synthetic dataset derived from ImageNet-1k, Hard-Spurious-ImageNet, which contains images with various backgrounds, object positions, and object sizes. By evaluating the dataset on different pretrained models, we find that most models rely heavily on spurious features in the background when the region-of-interest (ROI) to image ratio is small and the object is far from the center of the image. Moreover, we also show that current methods that aim to mitigate harmful spurious features, do not take into account these factors, hence fail to achieve considerable performance gains for worst-group accuracies when the size and location of core features in an image change. The dataset and implementation code are available at https://github.com/Mishalfatima/Corner_Cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。