arXiv:2501.07260cs.CVcs.AI2025-01AAAI被引 14

用单目图像实现高精度3D语义场景补全,性能超越现有单目方法。

Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion

  • 结合状态空间与扩散模型,在变分自编码器潜空间中建模
  • 提出Skimba去噪器,三向卷积+稀疏残差结构提升长序列处理效率
  • 在SemanticKITTI和SSCBench-KITTI360上表现优于主流单目方法

3D语义场景补全是自主系统中多项下游任务的关键。该任务需恢复采集数据中的缺失几何与语义信息。由于现实条件复杂,通常需处理多模态数据的复杂模型才能达到可接受性能。本文提出一种新型神经网络,融合状态空间模型与扩散生成建模技术,仅用单目图像即实现卓越的3D语义场景补全效果。方法在变分自编码器的条件潜空间中进行扩散建模,并引入创新的状态空间技术。核心组件为提出的Skimba(Skip Mamba)去噪器,擅长高效处理长序列数据。该扩散模型整合三重Mamba结构、维度分解残差及三方向可变膨胀。此外,还采用该网络变体完成后续语义分割阶段。在标准数据集SemanticKITTI和SSCBench-KITTI360上的大量实验表明,本方法不仅显著优于其他单目技术,且性能媲美立体方法。代码已开源:https://github.com/xrkong/skimba。

原文摘要 · Abstract (English)

3D semantic scene completion is critical for multiple downstream tasks in autonomous systems. It estimates missing geometric and semantic information in the acquired scene data. Due to the challenging real-world conditions, this task usually demands complex models that process multi-modal data to achieve acceptable performance. We propose a unique neural model, leveraging advances from the state space and diffusion generative modeling to achieve remarkable 3D semantic scene completion performance with monocular image input. Our technique processes the data in the conditioned latent space of a variational autoencoder where diffusion modeling is carried out with an innovative state space technique. A key component of our neural network is the proposed Skimba (Skip Mamba) denoiser, which is adept at efficiently processing long-sequence data. The Skimba diffusion model is integral to our 3D scene completion network, incorporating a triple Mamba structure, dimensional decomposition residuals and varying dilations along three directions. We also adopt a variant of this network for the subsequent semantic segmentation stage of our method. Extensive evaluation on the standard SemanticKITTI and SSCBench-KITTI360 datasets show that our approach not only outperforms other monocular techniques by a large margin, it also achieves competitive performance against stereo methods. The code is available at https://github.com/xrkong/skimba

3D补全单目图像扩散模型Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。