提升3D场景图预测,通过更精准的物体表征增强关系识别能力。
Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Prediction
- 设计可区分的物体特征编码器,用对比学习分离表征与预测任务。
- 在3DSSG上物体分类准确率提升,关系预测整体指标显著提高。
- 适合做3D场景理解、机器人感知和AR/VR应用的研究者使用。
3D语义场景图预测旨在检测3D场景中的物体及其语义关系,是机器人与增强现实/虚拟现实应用的关键技术。尽管先前研究解决了数据集局限性并探索了开放词汇设置,但普遍未能优化物体与关系特征的表征能力,过度依赖图神经网络却缺乏足够判别力。本文通过大量分析表明,物体特征质量对整体场景图准确率起决定性作用。为此,我们设计了一个高判别力的物体特征编码器,并采用对比预训练策略,将物体表征学习与场景图预测解耦。该设计不仅提升了物体分类准确率,还直接改善了关系预测性能。将预训练编码器接入现有框架后,所有评估指标均有显著提升。此外,我们有效融合几何与语义特征,充分利用关系信息。在3DSSG数据集上的全面实验表明,本方法显著优于此前最先进方法。代码已公开于https://github.com/VisualScienceLab-KHU/OCRL-3DSSG-Codes。
原文摘要 · Abstract (English)
3D Semantic Scene Graph Prediction aims to detect objects and their semantic relationships in 3D scenes, and has emerged as a crucial technology for robotics and AR/VR applications. While previous research has addressed dataset limitations and explored various approaches including Open-Vocabulary settings, they frequently fail to optimize the representational capacity of object and relationship features, showing excessive reliance on Graph Neural Networks despite insufficient discriminative capability. In this work, we demonstrate through extensive analysis that the quality of object features plays a critical role in determining overall scene graph accuracy. To address this challenge, we design a highly discriminative object feature encoder and employ a contrastive pretraining strategy that decouples object representation learning from the scene graph prediction. This design not only enhances object classification accuracy but also yields direct improvements in relationship prediction. Notably, when plugging in our pretrained encoder into existing frameworks, we observe substantial performance improvements across all evaluation metrics. Additionally, whereas existing approaches have not fully exploited the integration of relationship information, we effectively combine both geometric and semantic features to achieve superior relationship prediction. Comprehensive experiments on the 3DSSG dataset demonstrate that our approach significantly outperforms previous state-of-the-art methods. Our code is publicly available at https://github.com/VisualScienceLab-KHU/OCRL-3DSSG-Codes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。