让视觉语言模型按几何约束推理,解决空间判断不准的问题。
Geometrically-Constrained Agent for Spatial Reasoning
- 将模型分为语义分析与任务求解两阶段,用形式化约束限定空间推理
- 在多个基准上达到新最好性能,比现有方法提升约27%
- 无需训练,适合需要可靠空间推理的应用场景
视觉语言模型在空间推理中存在语义与几何之间的根本性差距:它们擅长定性语义推断,但推理过程在低保真语义空间中进行,与高精度几何不一致。当前范式无法弥合这一差距。基于训练的方法陷入‘预言悖论’,从有缺陷的标注中学习错误的空间逻辑;工具集成方法虽约束最终计算,却未限制模型的规划过程,导致几何错误的规划。本文提出几何约束智能体(GCA),一种无训练的代理范式,通过引入形式化任务约束来解决该问题。具体地,我们策略性地将视觉语言模型的角色拆分为两个阶段:第一阶段作为语义分析师,将用户模糊查询转化为形式化、可验证的任务约束,定义参考系与目标;第二阶段作为任务求解器,严格在约束所确定的确定性边界内生成并执行工具调用。这种几何约束推理策略有效弥合了语义到几何的鸿沟,实现鲁棒且可验证的空间推理路径。全面实验表明,GCA在多个空间推理基准上达到最先进水平,优于现有训练型和工具集成方法约27%。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) exhibit a fundamental semantic-to-geometric gap in spatial reasoning: they excel at qualitative semantic inference but their reasoning operates within a lossy semantic space, misaligned with high-fidelity geometry. Current paradigms fail to bridge this gap. Training-based methods suffer from an ``oracle paradox,'' learning flawed spatial logic from imperfect oracles. Tool-integrated methods constrain the final computation but critically leave the VLM's planning process unconstrained, resulting in geometrically flawed plans. In this work, we propose Geometrically-Constrained Agent (GCA), a training-free agentic paradigm that resolves this gap by introducing a formal task constraint. Specifically, we strategically decouples the VLM's role into two stages. First, acting as a semantic analyst, the VLM translates the user's ambiguous query into the formal, verifiable task constraint, which defines the reference frame and objective. Second, acting as a task solver, the VLM generates and executes tool calls strictly within the deterministic bounds defined by the constraint. This geometrically-constrained reasoning strategy successfully resolve the semantic-to-geometric gap, yielding a robust and verifiable reasoning pathway for spatial reasoning. Comprehensive experiments demonstrate that GCA achieves SOTA performance on multiple spatial reasoning benchmarks, surpassing existing training-based and tool-integrated methods by ~27%. Please see our homepage at https://gca-spatial-reasoning.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。