arXiv:2602.06037cs.CV2026-02被引 12

让大模型主动调用几何信息,提升空间推理能力

Thinking with Geometry: Active Geometry Integration for Spatial Reasoning

  • 模型根据推理需求主动检索几何证据,而非被动融合
  • 在VSI-Bench上达到72.6的最高分,超越现有方法
  • 适合需要精准空间理解的任务,如自动驾驶、机器人导航

多模态大模型在空间推理中的进展越来越多地利用3D编码器提供的几何先验。然而,现有集成策略多为被动方式:几何信息以全局流形式输入,并被无差别融合,常导致语义与几何错配及冗余信号。我们提出GeoThinker框架,将范式从被动融合转向主动感知。不同于特征混合,GeoThinker使模型能根据内部推理需求选择性检索几何证据。该机制通过在特定视觉语言模型(VLM)层应用空间锚定融合实现,其中语义视觉先验通过帧严格交叉注意力选择性查询并整合任务相关的几何信息,再经重要性门控校准,使每帧注意力偏向任务相关结构。全面评估显示,GeoThinker在空间智能任务上达到新基准,于VSI-Bench取得72.6的峰值得分。此外,其在复杂下游场景中展现出强泛化能力与显著提升的空间感知性能,包括具身指代和自动驾驶。结果表明,主动整合空间结构是下一代空间智能的关键。代码见https://github.com/Li-Hao-yuan/GeoThinker。

原文摘要 · Abstract (English)

Recent progress in spatial reasoning with Multimodal Large Language Models (MLLMs) increasingly leverages geometric priors from 3D encoders. However, most existing integration strategies remain passive: geometry is exposed as a global stream and fused in an indiscriminate manner, which often induces semantic-geometry misalignment and redundant signals. We propose GeoThinker, a framework that shifts the paradigm from passive fusion to active perception. Instead of feature mixing, GeoThinker enables the model to selectively retrieve geometric evidence conditioned on its internal reasoning demands. GeoThinker achieves this through Spatial-Grounded Fusion applied at carefully selected VLM layers, where semantic visual priors selectively query and integrate task-relevant geometry via frame-strict cross-attention, further calibrated by Importance Gating that biases per-frame attention toward task-relevant structures. Comprehensive evaluation results show that GeoThinker sets a new state-of-the-art in spatial intelligence, achieving a peak score of 72.6 on the VSI-Bench. Furthermore, GeoThinker demonstrates robust generalization and significantly improved spatial perception across complex downstream scenarios, including embodied referring and autonomous driving. Our results indicate that the ability to actively integrate spatial structures is essential for next-generation spatial intelligence. Code can be found at https://github.com/Li-Hao-yuan/GeoThinker.

空间推理几何先验主动感知多模态模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。