arXiv:2606.15328cs.CV2026-06

用新Transformer架构提升3D场景图生成,支持增量学习。

SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation

论文配图:SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation
图 1 · 摘自论文原文
  • 用Transformer实现全局信息传递,解决传统GCN过平滑问题。
  • 在3DSSG上增量设置下准确率提升4.49%。
  • 适合做3D场景理解与持续学习的科研人员。

本文提出SGFormer++,一种用于3D场景图生成(3D SGG)的新颖语义图Transformer模型,旨在将点云场景解析为包含物体实例节点与关系边的语义结构图。核心挑战在于建模复杂的全局场景结构。现有基于图卷积网络(GCN)的方法存在过平滑和感受野有限的问题,而SGFormer++采用Transformer作为主干,实现全局消息传递。具体引入两个关键组件:(1) 图嵌入层++,以线性复杂度高效融合边感知的全局上下文;(2) 语义注入层++,利用大语言模型(LLMs)和视觉-语言模型(VLMs)的语言先验增强视觉特征,不增加额外可训练参数。为应对增量3D SGG(I-SGG)中关系类别逐步到来的实际挑战,模型引入空间引导特征适配器,通过主体-客体空间几何校准谓词特征以缓解尺度变化,并设计级联二分类头,通过任务增量分类器扩展与对数蒸馏机制缓解灾难性遗忘。在3DSSG基准上的大量实验表明,SGFormer++在标准与增量设置下均达到最先进性能,增量设置下谓词准确率A@1提升4.49%绝对值。代码与数据已公开于https://github.com/Andy20178/SGFormer。

原文摘要 · Abstract (English)

In this paper, we propose SGFormer++, a novel Semantic Graph Transformer for 3D scene graph generation (SGG), which aims to parse point cloud scenes into semantic structural graphs, where nodes denote detected object instances and edges encode their pairwise relationships, with the core challenge lying in modeling complex global scene structure. While existing graph convolutional network (GCN)-based methods suffer from over-smoothing and limited receptive fields, SGFormer++ leverages Transformer layers as its backbone to enable global message passing. Specifically, we introduce two key components tailored for 3D SGG: (1) a Graph Embedding Layer++ that efficiently integrates edge-aware global context with linear computational complexity, and (2) a Semantic Injection Layer++ that enriches visual features with linguistic priors from large language models (LLMs) and vision-language models (VLMs), boosting semantic representation without introducing extra trainable parameters. To further address the practical challenge of incremental SGG (I-SGG), where new relationship categories arrive sequentially, we equip SGFormer++ with a novel Spatial-guided Feature Adapter, which calibrates predicate features using subject-object spatial geometry to counter scale variation, and a Cascaded Binary Prediction Head that mitigates catastrophic forgetting via task-incremental classifier expansion and logit distillation. Extensive experiments on the 3DSSG benchmark demonstrate that SGFormer++ achieves state-of-the-art performance in both standard and incremental settings: it yields a significant 4.49% absolute improvement in Predicate A@1 under the incremental setting. Code and data are available at: https://github.com/Andy20178/SGFormer.

3D场景图Transformer增量学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。