用结构化场景图匹配实现零样本3D视觉定位，提升空间一致性与可解释性。
SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching
- 通过2D视图推断物体关系并构建持久3D场景图
- 查询图与场景图约束对齐，实现多视角一致定位
- 适用于移动机器人长时程物理环境中的空间推理
零样本3D视觉定位需从自然语言中定位未结构化环境中的物体。现有视觉语言模型方法依赖视图相关推理或隐式表征，限制了组合查询下的空间一致性和可解释性。我们提出SceneGraphGrounder，将3D定位重构为对重建3D场景图的结构化图匹配。为此，引入视觉标记提示策略，使视觉语言模型从2D视图中推断物体间关系，并将其提升至包含空间与语义关系的持久3D场景图。给定查询后，构建查询图并与场景图进行约束对齐，确保多视图一致性与可解释推理。在ScanRefer基准测试中，本方法仅使用RGB-D输入即达到零样本方法中的竞争力表现。进一步通过移动端机器人实测验证，展示了在长时程物理环境中稳健的空间推理能力。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Zero-shot 3D visual grounding requires localizing objects in unstructured environments from free-form natural language. Recent vision-language model (VLM) approaches achieve promising results but rely on view-dependent reasoning or implicit representations, limiting spatial consistency and interpretability for compositional queries. We propose SceneGraphGrounder, a framework that reformulates 3D grounding as structured graph matching over a reconstructed 3D scene graph. To enable this formulation, we introduce a visual marker prompting strategy that enables a VLM to infer object-object relationships from 2D views, which are subsequently lifted into a persistent 3D scene graph encoding both spatial and semantic relations. Given a query, we construct a query graph and perform constrained alignment with the scene graph, ensuring multi-view consistency and interpretable reasoning. Experiments on the ScanRefer benchmark demonstrate that our method achieves competitive performance among zero-shot approaches, using only RGB-D inputs. We further validate our framework through real-world deployment on a mobile robot, demonstrating robust spatial reasoning in long-horizon physical environments. We will make our code publicly available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。