arXiv:2605.22558cs.CV2026-05

让视觉标记先获取精准几何信息,提升模型空间推理能力。

GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning

论文配图:GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning
图 1 · 摘自论文原文
  • 为每个视觉标记动态分配最相关的几何证据,而非统一使用
  • 在多个空间推理基准上显著提升几何感知能力,最高提升12.3%
  • 适合需要精确空间理解的视觉推理任务,如导航、场景解析

视觉语言模型中的时空推理需要保留物理几何而非仅依赖语义外观的视觉表征。现有多模态模型通过结构分支、3D感知监督、推理阶段融合或长时记忆引入几何信息,但通常将几何线索视为所有视觉标记共享的信号。本文指出,不同视觉标记因空间角色不同,所需几何证据也应各异。为此,提出GeoWeaver:一个预推理阶段的几何基础框架,从冻结的几何编码器构建多层级几何库,并对每个视觉标记进行自适应几何证据分配,使其在语言建模前通过残差接地操作融合最相关几何抽象。实验表明,该方法在多个空间推理基准上持续提升几何感知推理能力,同时保持通用多模态性能。结果表明,几何信息作为推理的基础前提比后期融合更有效。代码与模型将公开于https://github.com/yahooo-m/GeoWeaver。

原文摘要 · Abstract (English)

Spatio-temporal reasoning in vision-language models requires visual representations that preserve physical geometry rather than merely semantic appearance. Recent multimodal models incorporate geometric information through structural branches, 3D-aware supervision, reasoning-stage fusion, or long-horizon memory. While these approaches demonstrate the importance of geometry for spatial intelligence, they typically treat geometric cues as a shared signal across all visual tokens. We note that this overlooks a finer-grained challenge: different visual tokens require different geometric evidence depending on their spatial roles. To address this limitation, we introduce GeoWeaver, a pre-reasoning geometric grounding framework that treats geometry as a representational prerequisite for spatio-temporal reasoning. GeoWeaver constructs a multi-level geometry bank from a frozen geometry encoder and performs token-adaptive geometric evidence allocation, enabling each visual token to retrieve the most relevant geometric abstractions. The selected evidence is incorporated into visual tokens via a residual grounding operation prior to language modeling, yielding geometry-grounded representations for downstream reasoning. Extensive evaluations on spatial reasoning benchmarks demonstrate that GeoWeaver consistently enhances geometry-aware reasoning while retaining general multimodal capabilities. This indicates that geometric information yields the greatest benefit not as a late-fusion auxiliary signal but as a fundamental prerequisite that shapes the representational foundation on which large language models perform reasoning. All source code and models will be released at https://github.com/yahooo-m/GeoWeaver .

空间推理几何感知视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。