用扩散模型提升深度图像语义分割，更抗噪声且训练更快。
Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer
- 用可变形注意力变换器编码深度图，精准捕捉无效区域特征。
- 在NYUv2和SUN-RGBD上达最新性能，尤其在困难数据上表现突出。
- 生成式框架更善于建模数据分布，训练时间显著少于判别方法。
基于视觉的感知与推理对自主系统理解环境至关重要。RGB与深度图像常被用于捕获环境的语义与几何特征。开发可靠的方法来解析此类数据对真实应用极为关键，因噪声测量往往不可避免。本文提出一种基于扩散模型的RGB-D语义分割框架，并证明采用可变形注意力变换器作为编码器提取深度图特征，能有效捕捉深度测量中无效区域的特性。该生成式框架具备更强的数据分布建模能力，在复杂场景下表现稳健，且相比判别方法显著减少训练时间。实验表明，该方法在NYUv2与SUN-RGBD数据集上均达到当前最优性能，尤其在最具挑战性的图像数据上优势明显。
原文摘要 · Abstract (English)
Vision-based perception and reasoning is essential for scene understanding in any autonomous system. RGB and depth images are commonly used to capture both the semantic and geometric features of the environment. Developing methods to reliably interpret this data is critical for real-world applications, where noisy measurements are often unavoidable. In this work, we introduce a diffusion-based framework to address the RGB-D semantic segmentation problem. Additionally, we demonstrate that utilizing a Deformable Attention Transformer as the encoder to extract features from depth images effectively captures the characteristics of invalid regions in depth measurements. Our generative framework shows a greater capacity to model the underlying distribution of RGB-D images, achieving robust performance in challenging scenarios with significantly less training time compared to discriminative methods. Experimental results indicate that our approach achieves State-of-the-Art performance on both the NYUv2 and SUN-RGBD datasets in general and especially in the most challenging of their image data. Our project page will be available at https://diffusionmms.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。