针对3D语义场景补全中空洞多、前景少的问题,提出新网络提升精度。
Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion
- 用虚拟节点跳过空体素,仅聚焦有信息的体素进行特征优化。
- 在SemanticKITTI上达18.2% mIoU,优于已有单目和双目方法。
- 适合做自动驾驶场景理解的研究者参考,尤其关注稀疏数据建模。
单目语义场景补全(SSC)旨在从单张RGB图像重建完整的3D语义场景,为自动驾驶和机器人提供低成本解决方案。然而,体素分布固有的不平衡性——超过93%的体素为空,且前景类别稀少——带来了巨大挑战。现有方法常过度关注无信息体素,对长尾类别泛化能力差。为此,本文提出VoxSAMNet(体素稀疏感知调制网络),一个统一框架,显式建模体素稀疏性和语义不平衡性。提出:(1) 用于特征精炼的虚节点快捷路径(DSFR)模块,通过共享虚拟节点跳过空体素,同时使用可变形注意力优化占位体素;(2) 前景调制策略,结合前景丢弃(FD)与文本引导图像滤波(TGIF),缓解过拟合并增强类别相关特征。在公开基准SemanticKITTI和SSCBench-KITTI-360上的大量实验表明,VoxSAMNet达到当前最优性能,单目和双目基线的mIoU分别达18.2%和20.2%。结果凸显了稀疏感知与语义引导设计在高效准确的3D场景补全中的重要性,为未来研究提供了新方向。
原文摘要 · Abstract (English)
Monocular Semantic Scene Completion (SSC) aims to reconstruct complete 3D semantic scenes from a single RGB image, offering a cost-effective solution for autonomous driving and robotics. However, the inherently imbalanced nature of voxel distributions, where over 93% of voxels are empty and foreground classes are rare, poses significant challenges. Existing methods often suffer from redundant emphasis on uninformative voxels and poor generalization to long-tailed categories. To address these issues, we propose VoxSAMNet (Voxel Sparsity-Aware Modulation Network), a unified framework that explicitly models voxel sparsity and semantic imbalance. Our approach introduces: (1) a Dummy Shortcut for Feature Refinement (DSFR) module that bypasses empty voxels via a shared dummy node while refining occupied ones with deformable attention; and (2) a Foreground Modulation Strategy combining Foreground Dropout (FD) and Text-Guided Image Filter (TGIF) to alleviate overfitting and enhance class-relevant features. Extensive experiments on the public benchmarks SemanticKITTI and SSCBench-KITTI-360 demonstrate that VoxSAMNet achieves state-of-the-art performance, surpassing prior monocular and stereo baselines with mIoU scores of 18.2% and 20.2%, respectively. Our results highlight the importance of sparsity-aware and semantics-guided design for efficient and accurate 3D scene completion, offering a promising direction for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。