arXiv:2510.16688cs.CVcs.AI2025-10被引 2

让视觉语言模型更精准地理解3D空间,通过精简信息提升推理能力。

Pursuing Minimal Sufficiency in Spatial Reasoning

  • 构建最小必要信息集,仅保留回答问题所需的关键3D感知结果。
  • 在两个基准上达到顶尖性能,准确率显著优于现有方法。
  • 推理过程可解释,适合需要透明决策的高要求应用场景。

空间推理,即把语言与三维理解相结合的能力,仍是视觉语言模型(VLMs)的持久挑战。我们识别出两大根本瓶颈:源于二维中心预训练的三维理解不足,以及冗余三维信息引发的推理失败。为此,我们首先在回答问题前构建一个最小必要信息集(MSS)——从专家模型中精选的紧凑3D感知结果。我们提出MSSR(最小必要空间推理器),一种双代理框架来实现该原则。感知代理使用多功能感知工具箱程序化查询3D场景,提取充分信息,包括一种新颖的SOG(情境方向定位)模块,能鲁棒地提取语言引导的方向。推理代理则迭代优化这些信息,剔除冗余细节,并在闭环中请求缺失信息,直至形成完备的MSS。大量实验表明,通过显式追求充分性与最小性,该方法显著提升准确率,在两个挑战性基准上达到当前最优表现。此外,框架生成可解释的推理路径,为未来模型提供高质量训练数据。源代码已开源:https://github.com/gyj155/mssr。

原文摘要 · Abstract (English)

Spatial reasoning, the ability to ground language in 3D understanding, remains a persistent challenge for Vision-Language Models (VLMs). We identify two fundamental bottlenecks: inadequate 3D understanding capabilities stemming from 2D-centric pre-training, and reasoning failures induced by redundant 3D information. To address these, we first construct a Minimal Sufficient Set (MSS) of information before answering a given question: a compact selection of 3D perception results from \textit{expert models}. We introduce MSSR (Minimal Sufficient Spatial Reasoner), a dual-agent framework that implements this principle. A Perception Agent programmatically queries 3D scenes using a versatile perception toolbox to extract sufficient information, including a novel SOG (Situated Orientation Grounding) module that robustly extracts language-grounded directions. A Reasoning Agent then iteratively refines this information to pursue minimality, pruning redundant details and requesting missing ones in a closed loop until the MSS is curated. Extensive experiments demonstrate that our method, by explicitly pursuing both sufficiency and minimality, significantly improves accuracy and achieves state-of-the-art performance across two challenging benchmarks. Furthermore, our framework produces interpretable reasoning paths, offering a promising source of high-quality training data for future models. Source code is available at https://github.com/gyj155/mssr.

空间推理视觉语言模型可解释性3D理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。