将3D感知与空间推理分离,用现成模型+大模型推理提升空间智能效率
Disentangling 3D Modeling from Spatial Reasoning

- 用现成3D感知模型生成结构化几何证据,再让大模型基于此推理
- 在多个基准上表现媲美端到端模型,无需大规模3D训练数据
- 模块化设计提升可解释性与效率,适合需要透明决策的场景
本文提出一种新范式,将3D感知与空间推理显式解耦。现代感知模型擅长估计连续3D几何,而大语言模型(LLMs)在组合与符号推理方面表现优异。为此,我们设计了解耦空间推理器(DiSR),利用现成专家感知模型重建物理世界为结构化3D证据,并通过LoRA微调LLM仅基于这些显式几何证据进行推理。DiSR无需大规模3D视觉问答(VQA)训练或复杂工具使用策略,在主流空间推理基准上实现竞争力表现。除性能优异外,该框架还具备更高可解释性、模块化与计算效率,表明显式分离感知与推理是构建空间智能的可扩展且高效替代方案。
原文摘要 · Abstract (English)
In this work, we explore an alternative paradigm for spatial reasoning by explicitly disentangling 3D perception from reasoning, rather than jointly acquiring implicit 3D perception and reasoning through large-scale training. Our key observation is that modern perception models excel at estimating continuous 3D geometry, whereas large language models (LLMs) are particularly effective at compositional and symbolic reasoning. Motivated by these complementary strengths, we propose the Disentangled Spatial Reasoner (DiSR), a simple yet effective framework that reconstructs the physical world into structured 3D evidence using off-the-shelf expert perception models and fine-tunes an LLM with LoRA to perform reasoning solely over this explicit geometric evidence. Without large-scale 3D VQA training or complex tool-use policies, DiSR achieves competitive performance on popular spatial reasoning benchmarks. Beyond its strong performance, DiSR offers improved interpretability, modularity, and computational efficiency, demonstrating that explicit separation of perception and reasoning is a scalable and effective alternative paradigm to end-to-end modeling for spatial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。