研究视觉大模型在真实场景下的鲁棒性，揭示其应对光照、噪声等挑战的机制与局限。
An Investigation of Visual Foundation Models Robustness
- 分析视觉大模型在动态环境中的鲁棒性需求与防御策略
- 指出当前防御方法在分布偏移和对抗攻击下仍存不足
- 提供可复现的消融实验框架与评估指标，适合安全敏感领域研究者
视觉基础模型（VFMs）正广泛应用于计算机视觉领域，支撑目标检测、图像分类、分割、姿态估计和运动追踪等多种任务。它们依托深度学习里程碑模型如LeNet-5、AlexNet、ResNet、VGGNet、InceptionNet、DenseNet、YOLO和ViT，在生物识别、自动驾驶感知和医学影像分析等对安全性要求高的场景中表现优异。然而，这些系统在真实环境中面临光照变化、天气影响和传感器特性差异等动态挑战，鲁棒性成为建立用户信任的关键。本文系统研究了视觉系统在复杂环境下的鲁棒性需求，评估了主流经验性防御手段与鲁棒训练方法在应对分布偏移、噪声输入及空间畸变、对抗攻击方面的有效性。进一步分析了现有防御机制的局限性，包括网络结构特性与组件影响，并提出用于消融实验和基准测试的评估指标体系。
原文摘要 · Abstract (English)
Visual Foundation Models (VFMs) are becoming ubiquitous in computer vision, powering systems for diverse tasks such as object detection, image classification, segmentation, pose estimation, and motion tracking. VFMs are capitalizing on seminal innovations in deep learning models, such as LeNet-5, AlexNet, ResNet, VGGNet, InceptionNet, DenseNet, YOLO, and ViT, to deliver superior performance across a range of critical computer vision applications. These include security-sensitive domains like biometric verification, autonomous vehicle perception, and medical image analysis, where robustness is essential to fostering trust between technology and the end-users. This article investigates network robustness requirements crucial in computer vision systems to adapt effectively to dynamic environments influenced by factors such as lighting, weather conditions, and sensor characteristics. We examine the prevalent empirical defenses and robust training employed to enhance vision network robustness against real-world challenges such as distributional shifts, noisy and spatially distorted inputs, and adversarial attacks. Subsequently, we provide a comprehensive analysis of the challenges associated with these defense mechanisms, including network properties and components to guide ablation studies and benchmarking metrics to evaluate network robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。