让3D大模型分步推理空间关系，精准定位复杂描述中的物体。
SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs
- 用隐式推理步骤逐步分析空间关系，替代单次选择
- 在多数据集上显著提升细粒度定位准确率，优于现有统一模型
- 适合需要精确空间理解的3D视觉任务，如机器人导航
3D物体定位需根据自然语言描述在3D场景中找到对应物体。统一实例中心型3D-LLM虽能同时处理定位、对话与问答，但多数依赖单次指针式决策，将复杂关系压缩为单一选择，对细粒度查询表现脆弱。本文提出结构化空间推理3D-LLM（SSR3D-LLM），基于固定Mask3D物体候选，让大模型生成一系列隐式空间推理步骤与记忆标记，由几何感知评分器按序读取并逐步优化候选排序，通过步骤长度掩码控制。训练时使用标准基准监督及辅助指代线索监督，推理仅需输入查询与Mask3D提案。在ReferIt3D、ScanRefer和Multi3DRef上，SSR3D-LLM超越所有统一3D-LLM基线，对细粒度定位的性能提升显著，且保持原有语言任务路径不变。
原文摘要 · Abstract (English)
3D object grounding localizes referred objects in a 3D scene from natural language. Unified instance-centric 3D-LLMs aim to solve grounding together with dialog, QA, and captioning, yet many rely on a single pointer-style grounding decision that compresses a relational instruction into one selection. This is brittle for fine-grained queries where multiple same-class candidates must be ruled out by context objects and spatial relations. We propose Structured Spatial Reasoning 3D-LLM (SSR3D-LLM), a structured grounding interface for unified 3D-LLMs. Given fixed Mask3D object proposals, the LLM writes a sequence of latent spatial reasoning steps and memory tokens from the query, and a geometry-aware scorer reads these latent steps in order to refine candidate rankings step by step with step-length masking. The latent steps are learned from standard benchmark target supervision with auxiliary referential-cue supervision during training, while inference uses only the input query and Mask3D proposals. Across ReferIt3D, ScanRefer, and Multi3DRef, SSR3D-LLM achieves the strongest results among unified 3D-LLM baselines, with substantial gains over the single-pointer QPG baseline on fine-grained grounding and consistent improvements over prior unified 3D-LLMs, while preserving the default language-task route.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。