arXiv:2605.20549cs.CV2026-05被引 1

构建可控3D场景数据集,揭示视觉模型预测依赖的关键因素。

MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space

论文配图:MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space
图 1 · 摘自论文原文
  • 设计可调控的9个3D场景参数,通过渲染生成可控图像。
  • 发现相机距离与俯仰角是导致识别失败的核心因素。
  • 适合研究模型鲁棒性、可解释性及架构设计的科研人员。

当前视觉模型在标准基准上表现优异,但其整体准确率难以揭示驱动预测的具体场景属性。现有鲁棒性测试通常操控全局2D图像属性,依赖纠缠的真实世界变化,或仅覆盖有限的3D物体与场景参数。本文提出MAPS(Manifolds of Artificial Parametric Scenes),一个可扩展的可控工具,用于将视觉模型行为归因于具体场景参数。MAPS包含2,618个经验证的逼真3D网格,覆盖560个ImageNet类别,并提供基于Blender的渲染流水线,支持背景、相机、光照等9个独立场景因子的连续变化,可扩展至其他因子。为验证其应用价值,我们使用MAPS评估20个卷积与基于Transformer的模型,通过回归敏感性分析量化其对这些场景因子的依赖程度。结果发现所有架构均存在近乎普遍的失败轴:相机距离与俯仰角始终主导识别失败,无论ImageNet准确率高低。然而,完整敏感性结构显示,现代CNN与Transformer聚类在一起,区别于旧架构,表明精细的架构设计选择比粗粒度的CNN与Transformer区分更能决定敏感性模式。

原文摘要 · Abstract (English)

Modern vision models achieve strong performance on standard benchmarks, yet their aggregate accuracy reveals little about which scene properties drive their predictions. Existing robustness benchmarks provide important stress tests, but typically manipulate global 2D image properties, rely on entangled real-world variation, or cover only a limited set of 3D objects and scene parameters. We introduce MAPS (Manifolds of Artificial Parametric Scenes), a scalable instrument for controlled attribution of vision model behavior to scene parameters. MAPS comprises 2,618 curated photorealistic 3D meshes validated for recognizability across 560 ImageNet classes and provides a Blender-based rendering pipeline for on-demand image generation under continuous variation of nine independent scene-factors spanning background, camera, and lighting, extensible to other factors. To showcase its applicability, we use MAPS to evaluate 20 convolutional and transformer-based models by quantifying their reliance on these scene factors through regression-based sensitivity analysis. We find a near-universal failure axis across all tested architectures: camera distance and elevation consistently dominate recognition failure regardless of ImageNet accuracy. However, the full sensitivity structure reveals that modern CNNs and transformers cluster together, distinct from older architectures, suggesting that fine-grained architectural design choices, rather than the coarse CNN-versus-transformer distinction, are the stronger determinant of sensitivity profiles.

视觉模型3D场景可解释性数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。