提出兼顾时空对称性的3D场景图网络，提升多视角理解的精度与稳定性。
TESGNN: Temporal Equivariant Scene Graph Neural Networks for Efficient and Robust Multi-View 3D Scene Understanding
- 基于对称性保持的图神经网络提取点云关系，生成更稳定的场景图
- 通过近似图匹配融合多时序场景图，实现动态实体关系建模
- 计算高效，适合机器人与视觉领域的实时应用
场景图因其紧凑且显式的关联信息表达，在各类场景理解任务中表现优异。然而，现有方法常忽略从三维点云生成场景图时保持对称性的重要性，导致在噪声多、多视角数据下准确率和鲁棒性下降。此外，以往方法缺乏对动态演化实体间时间依赖关系的建模能力。为此，我们提出时间等变场景图神经网络（TESGNN），包含两个核心组件：(1) 等变场景图神经网络（ESGNN），从点云中提取信息生成场景图，同时保持关键对称性；(2) 时间图匹配网络，利用近似图匹配算法将ESGNN在多个时间序列上生成的场景图融合为统一全局表示。实验表明，TESGNN在场景图生成任务中优于现有方法，具有更高准确率和更快训练收敛速度。更重要的是，借助对称性保持机制，其生成的全局场景表示更稳定、更准确。该模型计算高效，可基于现有框架轻松实现，适用于机器人与计算机视觉中的实时应用。代码已开源：https://github.com/HySonLab/TESGraph
原文摘要 · Abstract (English)
Scene graphs have proven to be highly effective for various scene understanding tasks due to their compact and explicit representation of relational information. However, current methods often overlook the critical importance of preserving symmetry when generating scene graphs from 3D point clouds, which can lead to reduced accuracy and robustness, particularly when dealing with noisy, multi-view data. Furthermore, a major limitation of prior approaches is the lack of temporal modeling to capture time-dependent relationships among dynamically evolving entities in a scene. To address these challenges, we propose Temporal Equivariant Scene Graph Neural Network (TESGNN), consisting of two key components: (1) an Equivariant Scene Graph Neural Network (ESGNN), which extracts information from 3D point clouds to generate scene graph while preserving crucial symmetry properties, and (2) a Temporal Graph Matching Network, which fuses scene graphs generated by ESGNN across multiple time sequences into a unified global representation using an approximate graph-matching algorithm. Our combined architecture TESGNN shown to be effective compared to existing methods in scene graph generation, achieving higher accuracy and faster training convergence. Moreover, we show that leveraging the symmetry-preserving property produces a more stable and accurate global scene representation compared to existing approaches. Finally, it is computationally efficient and easily implementable using existing frameworks, making it well-suited for real-time applications in robotics and computer vision. This approach paves the way for more robust and scalable solutions to complex multi-view scene understanding challenges. Our source code is publicly available at: https://github.com/HySonLab/TESGraph
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。