让视觉语言模型学会精准3D空间推理,突破传统二维局限。
Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regression
- 分离3D推理与数值生成,用专门模块处理空间逻辑和连续值输出。
- 1.5B模型在3D推理上媲美70亿以上参数模型,精度显著提升。
- 适合需要高精度空间理解的机器人、自动驾驶等应用。
现有视觉语言模型(VLM)基于“平面感知”架构,难以理解真实世界的3D空间智能。根源在于双重瓶颈:输入阶段,几何感知编码器计算开销大,而2D特征浅显;输出阶段,离散分词器无法生成精确连续数值。为此,我们提出GEODE(几何输出与解耦输入引擎),通过解耦3D推理与数值生成,解决双重瓶颈。GEODE在主VLM基础上引入两个即插即用模块:解耦推理模块(DRM)作为空间协处理器,通过交叉注意力将显式3D数据与2D视觉特征对齐,并提炼出可注入的3D链式思维(CoT)逻辑令牌;直接回归头(DRH)采用“嵌入即值”范式,将专用控制令牌送入轻量MLP,实现标量和3D边界框的精确连续回归。二者协同使我们的15亿参数模型成为高层语义调度器,在空间推理性能上达到领先水平,媲美70亿以上参数模型。
原文摘要 · Abstract (English)
Existing Vision Language Models (VLMs) architecturally rooted in "flatland" perception, fundamentally struggle to comprehend real-world 3D spatial intelligence. This failure stems from a dual-bottleneck: input-stage conflict between computationally exorbitant geometric-aware encoders and superficial 2D-only features, and output-stage misalignment where discrete tokenizers are structurally incapable of producing precise, continuous numerical values. To break this impasse, we introduce GEODE (Geometric-Output and Decoupled-Input Engine), a novel architecture that resolves this dual-bottleneck by decoupling 3D reasoning from numerical generation. GEODE augments main VLM with two specialized, plug-and-play modules: Decoupled Rationale Module (DRM) that acts as spatial co-processor, aligning explicit 3D data with 2D visual features via cross-attention and distilling spatial Chain-of-Thought (CoT) logic into injectable Rationale Tokens; and Direct Regression Head (DRH), an "Embedding-as-Value" paradigm which routes specialized control tokens to a lightweight MLP for precise, continuous regression of scalars and 3D bounding boxes. The synergy of these modules allows our 1.5B parameter model to function as a high-level semantic dispatcher, achieving state-of-the-art spatial reasoning performance that rivals 7B+ models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。