让3D场景图随时间增量更新，融合历史观测与多模态信息。
Integrating Prior Observations for Incremental 3D Scene Graph Prediction
- 构建异构图模型，在消息传递中融入历史观测与语义嵌入
- 在3DSSG数据集上实现可扩展的增量预测性能
- 适合机器人、具身智能等动态环境应用
3D语义场景图（3DSSG）通过显式建模物体、属性和关系，提供环境的紧凑结构化表示。尽管3DSSG在机器人和具身AI中展现潜力，现有方法主要依赖传感器数据，未能整合语义丰富的环境信息。此外，多数方法假设可获得完整场景重建，限制了其在真实世界增量场景中的应用。本文提出一种新颖的异构图模型，用于增量3DSSG预测，直接将先验观测等多模态信息融入消息传递过程。该模型通过多层设计，灵活融合全局与局部场景表征，无需专用模块或完整场景重建。我们在3DSSG数据集上评估该方法，结果表明，结合语义嵌入（如CLIP）和先验观测的图神经网络，为复杂真实环境提供了可扩展且通用的解决方案。完整源代码将公开于 https://github.com/m4renz/incremental-scene-graph-prediction。
原文摘要 · Abstract (English)
3D semantic scene graphs (3DSSG) provide compact structured representations of environments by explicitly modeling objects, attributes, and relationships. While 3DSSGs have shown promise in robotics and embodied AI, many existing methods rely mainly on sensor data, not integrating further information from semantically rich environments. Additionally, most methods assume access to complete scene reconstructions, limiting their applicability in real-world, incremental settings. This paper introduces a novel heterogeneous graph model for incremental 3DSSG prediction that integrates additional, multi-modal information, such as prior observations, directly into the message-passing process. Utilizing multiple layers, the model flexibly incorporates global and local scene representations without requiring specialized modules or full scene reconstructions. We evaluate our approach on the 3DSSG dataset, showing that GNNs enriched with multi-modal information such as semantic embeddings (e.g., CLIP) and prior observations offer a scalable and generalizable solution for complex, real-world environments. The full source code of the presented architecture will be made available at https://github.com/m4renz/incremental-scene-graph-prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。