用文字描述深度顺序，比给图像更有效提升视觉语言模型的空间推理能力。
When Depth Is Better Told Than Shown: Depth-Ordinal Prompting for Vision-Language Spatial Reasoning
- 将单目深度转为针对目标物体的序数文本提示，无需额外训练或图像输入。
- 在多个基准上，当伪深度提供可靠物体排序时，空间推理准确率显著提升。
- 适合需要提升空间理解但无法修改模型结构的研究者使用。
视觉语言模型（VLMs）应能进行物理空间推理——判断哪个物体更近、什么在什么背后、物体如何三维排列——但依然存在困难。直接提供深度图反而可能降低性能。我们发现深度信息虽已传入语言模型，但难以用于下游推理；而伪深度图作为噪声辅助图像，冻结的VLM难以调节。为此提出深度序数提示（DOP），一种无需训练的方法：将单目深度转化为针对查询物体的单一序数文本提示，不添加深度图像、不训练模块、不注入特征、不依赖标签。关键发现是形式依赖性：相同深度信号以图像形式呈现会损害性能，以文本形式表达则能提升表现。在多个基准、模型和深度估计算法下，当伪深度提供可靠物体层级排序时，DOP显著改善空间推理；在原始图像表现强劲的场景中则基本保持中立。其效果与最强的无训练深度提示方法相当，但更简单、更精准。
原文摘要 · Abstract (English)
Vision-language models (VLMs) are expected to reason about physical space -- which object is closer, what lies behind what, and how objects are arranged in 3D -- yet they still struggle with such spatial judgments. A natural remedy is to show the model a depth map, but we find that this can make performance worse. We show that depth is not absent: it reaches the language model, but becomes difficult to access for downstream reasoning, while rendered pseudo-depth maps act as noisy auxiliary images that frozen VLMs cannot easily regulate. We propose Depth-Ordinal Prompting (DOP), a training-free method that converts monocular depth into a single question-targeted ordinal text cue at the queried objects, without adding a depth image, training a module, injecting features, or using labels. Our key finding is form dependence: the same depth signal can hurt when shown as an image but help when told as text.Across benchmarks, models, and depth estimators, DOP improves spatial reasoning when pseudo-depth provides reliable object-level ordering and remains largely neutral in strong original-image regimes. It is also competitive with the strongest training-free depth-prompting alternative while being simpler and more targeted.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。