arXiv:2602.00637cs.CV2026-02被引 2

无需训练,直接从3D场景生成视角不变的语义关系图

VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning

  • 基于物体朝向定义空间关系,避免视角变化导致歧义
  • 零样本生成开放词汇关系,在Replica和Nr3D上提升22%与4.81%准确率
  • 适用于需要跨视角一致理解的3D场景推理任务

3D场景理解与推理是计算机视觉中的基础问题,要求模型识别物体、属性及其空间或比较关系。现有方法依赖多源输入(如2D图像、深度图、对象标签和参考视角的标注关系),但普遍存在泛化能力差、空间关系(如左右)不一致的问题。为此,我们提出视角不变的零样本3D场景图生成框架VIZOR。该方法无需训练,端到端地从原始3D场景构建密集且视角无关的场景图。其空间关系以物体正面方向为基准定义,确保不同视角下一致性。同时,它可推断开放词汇的关系(如位置与邻近性),无需标注数据。我们在多个定量与定性评估中验证了VIZOR的有效性,其在场景图生成及下游任务(如基于查询的对象定位)中表现优异,在Replica和Nr3D数据集上的零样本定位准确率分别提升22%和4.81%。

原文摘要 · Abstract (English)

Scene understanding and reasoning has been a fundamental problem in 3D computer vision, requiring models to identify objects, their properties, and spatial or comparative relationships among the objects. Existing approaches enable this by creating scene graphs using multiple inputs such as 2D images, depth maps, object labels, and annotated relationships from specific reference view. However, these methods often struggle with generalization and produce inaccurate spatial relationships like "left/right", which become inconsistent across different viewpoints. To address these limitations, we propose Viewpoint-Invariant Zero-shot scene graph generation for 3D scene Reasoning (VIZOR). VIZOR is a training-free, end-to-end framework that constructs dense, viewpoint-invariant 3D scene graphs directly from raw 3D scenes. The generated scene graph is unambiguous, as spatial relationships are defined relative to each object's front-facing direction, making them consistent regardless of the reference view. Furthermore, it infers open-vocabulary relationships that describe spatial and proximity relationships among scene objects without requiring annotated training data. We conduct extensive quantitative and qualitative evaluations to assess the effectiveness of VIZOR in scene graph generation and downstream tasks, such as query-based object grounding. VIZOR outperforms state-of-the-art methods, showing clear improvements in scene graph generation and achieving 22% and 4.81% gains in zero-shot grounding accuracy on the Replica and Nr3D datasets, respectively.

3D理解场景图零样本视角不变

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。