通过解耦实例与场景上下文,提升3D语义场景补全精度
Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion
- 用类别查询替代体素查询,融合类别特有几何语义先验
- 在SemanticKITTI和SSCBench上达到17.35和20.55的mIoU新高
- 单帧输入超越多帧方法,尤其显著提升实例类别识别性能
3D语义场景补全(SSC)在三维感知中扮演关键角色,近年研究主要聚焦于优化体素级特征以重建3D场景。然而,将体素作为基本交互单元限制了类别级信息的利用,而该信息对提升补全结果粒度至关重要。为此,我们提出一种新颖的双流范式DISC(解耦实例与场景上下文),通过分离优化增强对实例和场景类别的学习。具体而言,我们用包含类别特有几何与语义先验的判别性类别查询替代体素查询,并利用类别内在特性设计专用解码模块,实现针对性交互与高效类别级信息流动。实验表明,DISC在SemanticKITTI和SSCBench-KITTI-360基准上分别取得17.35和20.55的mIoU,为当前最优表现。尤为突出的是,仅使用单帧输入即超越多帧现有方法,在SemanticKITTI隐藏测试集上实例mIoU相比单帧和多帧SOTA分别提升17.9%和11.9%。代码已开源。
原文摘要 · Abstract (English)
3D Semantic Scene Completion (SSC) has gained increasing attention due to its pivotal role in 3D perception. Recent advancements have primarily focused on refining voxel-level features to construct 3D scenes. However, treating voxels as the basic interaction units inherently limits the utilization of class-level information, which is proven critical for enhancing the granularity of completion results. To address this, we propose \textbf{D}isentangling Instance and Scene Contexts (DISC), a novel dual-stream paradigm that enhances learning for both instance and scene categories through separated optimization. Specifically, we replace voxel queries with discriminative class queries, which incorporate class-specific geometric and semantic priors. Additionally, we exploit the intrinsic properties of classes to design specialized decoding modules, facilitating targeted interactions and efficient class-level information flow. Experimental results demonstrate that DISC achieves state-of-the-art (SOTA) performance on both SemanticKITTI and SSCBench-KITTI-360 benchmarks, with mIoU scores of 17.35 and 20.55, respectively. Remarkably, DISC even outperforms multi-frame SOTA methods using only single-frame input and significantly improves instance category performance, surpassing both single-frame and multi-frame SOTA instance mIoU by 17.9\% and 11.9\%, respectively, on the SemanticKITTI hidden test. The code is available at https://github.com/Enyu-Liu/DISC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。