arXiv:2605.30161cs.CV2026-05被引 1

发现视觉语言模型的空间理解依赖透视偏见,而非真实三维认知。

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

论文配图:Why Far Looks Up: Probing Spatial Representation in Vision-Language Models
图 1 · 摘自论文原文
  • 通过构建最小对比对分析模型嵌入中的空间轴结构
  • 垂直位置与距离高度纠缠,导致在反直觉样本上准确率下降30%以上
  • 新基准SpatialTunnel可暴露模型内在空间偏差,适合研究鲁棒性

视觉语言模型在空间推理任务中表现优异,但其是否具备结构化3D理解仍不明确。本文提出一种表示层分析框架,通过构造最小对比对测量模型嵌入中空间轴的组织与解耦程度。跨多个模型家族的分析显示,存在一致的垂直距离纠缠现象:模型将图像垂直位置与距离混淆,反映出自然图像的透视偏见。这一偏见导致视角一致与反直觉样本间准确率差距显著,在数据量扩大时甚至加剧,尽管整体基准准确率提升。我们还发现,相似基准分数的模型内部表示差异明显,且这些差异能预测其在多样化空间推理任务中的准确率与鲁棒性。为隔离评估集偏差,我们引入Synthetic benchmark SpatialTunnel,通过移除自然图像中的常见相关性来暴露空间捷径偏差。实验表明该纠缠为模型固有特性,空间轴分离良好的模型表现出更强鲁棒性,提示良好结构化的空间表示有助于提升跨任务可靠性。代码与基准已公开于项目页面。

原文摘要 · Abstract (English)

Vision-language models (VLMs) achieve strong performance on spatial reasoning benchmarks, yet it remains unclear whether this reflects structured 3D understanding or reliance on statistical shortcuts in natural images. We introduce a representation-level analysis framework that constructs minimal contrastive pairs to measure how spatial axes are organized and disentangled within VLM embeddings. Our analysis across multiple model families reveals a consistent vertical-distance entanglement: models conflate vertical image position with distance, mirroring the perspective bias of natural photographs. This bias produces a significant accuracy gap between perspective-consistent and counter-heuristic examples, and intensifies under data scaling even as overall benchmark accuracy improves. We further show that models with similar benchmark scores can exhibit different internal representations, and that these differences predict accuracy and robustness across diverse spatial reasoning benchmarks. To isolate this bias from evaluation-set skew, we introduce SpatialTunnel, a synthetic benchmark designed to expose spatial shortcut biases by removing common correlations present in natural images. Experiments confirm that the entanglement is model-intrinsic, and that models with well-separated spatial axes exhibit greater robustness, suggesting that well-structured spatial representations lead to more reliable spatial reasoning across diverse benchmarks. Code and benchmark are available on the project page: https://cheolhong0916.github.io/whyfarlooksup.github.io/.

空间推理视觉语言模型表示分析鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。