提出观察者感知关系建模,让3D场景理解更适应视角变化。
From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation

- 区分依赖视角与不依赖视角的关系,设计观察者感知编码
- 在3DSSG数据集上,视角变换下性能稳定优于基线方法
- 适合需要多视角鲁棒性的3D场景理解任务
3D场景图生成(3DSGG)将3D场景表示为结构化的物体-关系-物体图,以实现空间理解。在观察者中心的空间感知中,同一场景可因局部观察者视角不同而表达为不同框架,但其结构保持不变。然而,现有模型通常假设固定的场景对齐参考帧,在场景以不同观察者框架重表达时,可能产生语义不一致的预测。我们归因于关系谓词的异质性框架依赖性:方向类谓词如左、前、右、后为观察者依赖关系,而多数接触、支撑和语义谓词如‘站在…上’‘附着于’则近似为观察者独立关系。传统模型未区分此类响应差异,导致观察者框架旋转时关系预测性能下降。本文提出观察者感知关系(OAR),结合观察者感知几何编码与关系专门化,并由框架稳定物体编码支持,实现统一的多标签谓词预测。在3DSSG上的实验表明,OAR在无训练时框架旋转增强的情况下,仍能持续优于基线方法,同时在标准基准上保持竞争力。
原文摘要 · Abstract (English)
3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object--relation--object graphs for spatial understanding. In observer-centric spatial perception, the same scene may be expressed under different local observer frames while its structure remains unchanged. However, existing models typically assume a fixed scene-aligned reference frame and may produce semantically inconsistent predictions when the scene is re-expressed in another observer frame. We attribute this failure to the heterogeneous frame dependency of relational predicates. Directional predicates such as $\textit{left}$, $\textit{front}$, $\textit{right}$, and $\textit{behind}$ are $\textbf{Observer-Dependent Relations}$, whereas most contact, support, and semantic predicates, such as $\textit{standing on}$ and $\textit{attached to}$, are approximately $\textbf{Observer-Independent Relations}$. Conventional models do not distinguish these frame responses, leading to degraded relation prediction under observer-frame reorientation. We introduce $\textbf{Observer-Aware Relations (OAR)}$, which combines observer-aware geometric encoding and relation specialization, supported by frame-stable object encoding, for unified multi-label predicate prediction. Experiments on 3DSSG show that OAR consistently outperforms baselines across controlled observer-frame reorientations without training-time frame-reorientation augmentation, while remaining competitive on the standard benchmark. The project page is available at https://oar-predicate.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。