arXiv:2601.05172cs.CVcs.AI2026-01ACL被引 9

让视觉语言模型主动切换视角,提升3D环境问答的空间推理能力。

CoV: Chain-of-View Prompting for Spatial Reasoning

  • 通过粗到精的视角探索策略,动态选择关键视角
  • 在OpenEQA上平均提升11.56%,最高达13.62%准确率
  • 无需训练,适配多种模型,适合3D场景智能应用

在3D环境中进行具身问答(EQA)常需整合分散于多视角且部分遮挡的上下文信息。然而,现有视觉-语言模型(VLMs)受限于固定有限的输入视角,难以在推理时获取问题相关上下文,阻碍复杂空间推理。本文提出链式视角(CoV)提示框架,一种无需训练、可在测试时执行的推理机制,使VLM具备主动视角探索能力。CoV首先由视角选择代理筛选冗余帧并定位与问题对齐的锚点视角;随后通过迭代推理与离散相机动作交替,逐步调整视角,从底层3D场景中获取新观测,直至获得充分上下文或达到步数预算。我们在OpenEQA上评估了四种主流VLM,平均提升LLM-Match 11.56%,最大提升达13.62%(Qwen3-VL-Flash)。CoV还表现出测试时扩展性:增加最小动作预算可带来额外+2.51%平均提升,峰值+3.73%(Gemini-2.5-Flash)。在ScanQA和SQA3D上表现优异(如ScanQA:116 CIDEr / 31.9 EM@1;SQA3D:51.1 EM@1)。结果表明,问题对齐的视角选择结合开放视角搜索,是一种无需额外训练、通用有效的空间推理增强策略。代码已开源。

原文摘要 · Abstract (English)

Embodied question answering (EQA) in 3D environments often requires collecting context that is distributed across multiple viewpoints and partially occluded. However, most recent vision--language models (VLMs) are constrained to a fixed and finite set of input views, which limits their ability to acquire question-relevant context at inference time and hinders complex spatial reasoning. We propose Chain-of-View (CoV) prompting, a training-free, test-time reasoning framework that transforms a VLM into an active viewpoint reasoner through a coarse-to-fine exploration process. CoV first employs a View Selection agent to filter redundant frames and identify question-aligned anchor views. It then performs fine-grained view adjustment by interleaving iterative reasoning with discrete camera actions, obtaining new observations from the underlying 3D scene representation until sufficient context is gathered or a step budget is reached. We evaluate CoV on OpenEQA across four mainstream VLMs and obtain an average +11.56% improvement in LLM-Match, with a maximum gain of +13.62% on Qwen3-VL-Flash. CoV further exhibits test-time scaling: increasing the minimum action budget yields an additional +2.51% average improvement, peaking at +3.73% on Gemini-2.5-Flash. On ScanQA and SQA3D, CoV delivers strong performance (e.g., 116 CIDEr / 31.9 EM@1 on ScanQA and 51.1 EM@1 on SQA3D). Overall, these results suggest that question-aligned view selection coupled with open-view search is an effective, model-agnostic strategy for improving spatial reasoning in 3D EQA without additional training. Code is available on https://github.com/ziplab/CoV .

空间推理视角探索具身问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。