arXiv:2409.10775cs.CVcs.AI2024-09被引 9

对比真实与人工遮挡,发现ViT模型在部分遮挡下表现接近人类。

Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?

  • 构建真实遮挡数据集IRUO,含自然与人工遮挡图像
  • ViT模型在遮挡下准确率优于CNN,仅略逊于人类
  • 透过缝隙等模糊遮挡会显著降低CNN模型性能

图像分类模型(包括CNN)在部分遮挡条件下表现下降。现有改进方法如数据增强、基于部件的聚类和更鲁棒的架构(如视觉变压器ViT)虽被评估,但多基于人工生成的遮挡图像,且常与过时模型比较。本文提出基于OVIS数据集的图像识别遮挡(IRUO)数据集,包含真实世界与人工遮挡图像,用于测试主流方法在视觉识别任务中对部分遮挡的鲁棒性。同时开展人类实验,评估不同遮挡程度下的分类表现。结果表明:现代CNN模型在遮挡图像上较早期模型有提升;ViT模型优于CNN模型,其准确率仅略低于人类水平。此外,通过栅栏、树叶等产生‘孔洞’式扩散遮挡时,深度模型性能显著下降,尤其以CNN为主模型者。

原文摘要 · Abstract (English)

Image classification models, including convolutional neural networks (CNNs), perform well on a variety of classification tasks but struggle under conditions of partial occlusion, i.e., conditions in which objects are partially covered from the view of a camera. Methods to improve performance under occlusion, including data augmentation, part-based clustering, and more inherently robust architectures, including Vision Transformer (ViT) models, have, to some extent, been evaluated on their ability to classify objects under partial occlusion. However, evaluations of these methods have largely relied on images containing artificial occlusion, which are typically computer-generated and therefore inexpensive to label. Additionally, methods are rarely compared against each other, and many methods are compared against early, now outdated, deep learning models. We contribute the Image Recognition Under Occlusion (IRUO) dataset, based on the recently developed Occluded Video Instance Segmentation (OVIS) dataset (arXiv:2102.01558). IRUO utilizes real-world and artificially occluded images to test and benchmark leading methods' robustness to partial occlusion in visual recognition tasks. In addition, we contribute the design and results of a human study using images from IRUO that evaluates human classification performance at multiple levels and types of occlusion. We find that modern CNN-based models show improved recognition accuracy on occluded images compared to earlier CNN-based models, and ViT-based models are more accurate than CNN-based models on occluded images, performing only modestly worse than human accuracy. We also find that certain types of occlusion, including diffuse occlusion, where relevant objects are seen through "holes" in occluders such as fences and leaves, can greatly reduce the accuracy of deep recognition models as compared to humans, especially those with CNN backbones.

视觉识别遮挡鲁棒性ViT人类对比

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。