用稀疏涂鸦生成精细伪标注,结合状态空间扩散模型提升RGB-D显著性检测精度
Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion

- 利用SAM将涂鸦扩展为密集伪标注,通过双分支结构保持分割一致性
- 提出S²Diff模型,通过条件特征融合与上下文注入实现噪声抑制和细节增强
- 在7个数据集上超越弱监督方法,接近全监督性能,适合标注成本高的场景
弱监督RGB-D显著性检测旨在减轻像素级标注的负担。但涂鸦标注缺乏物体结构与细节,导致显著图不准确。本文提出一种新型涂鸦监督方法,包含基于SAM的伪标注生成模块(SAM-PAG)和基于状态空间交互的条件扩散模型(S²Diff)。SAM-PAG采用先进SAM,通过双分支结构与分割掩码一致性,将稀疏涂鸦扩展为密集像素级伪标注。S²Diff利用扩散模型,在条件信息引导下迭代优化含噪显著图,生成高精度结果。其核心在于条件特征获取与去噪:前者通过频域融合与隐式-显式状态空间交互实现跨模态特征交织;后者通过上下文注入模块抑制噪声并强化对象信息。SAM-PAG与S²Diff协同工作,在七个数据集上优于现有弱监督方法,并达到与全监督方法相当的性能。代码与结果见https://github.com/Switch457/WeakS2Diff_SOD。
原文摘要 · Abstract (English)
Weakly-supervised RGB-D Salient Object Detection (SOD) is explored to reduce the heavy burden of pixel-level annotations. But scribble annotations lack the structure and details of objects, resulting in inaccurate saliency maps. In this paper, we propose a novel scribble-supervised RGB-D SOD method, consisting of a Segment Anything Model (SAM)-driven pseudo annotation generation method (\emph{SAM-PAG}) and a state space interaction-based conditional diffusion model (\emph{$S^2$Diff}). Specifically, SAM-PAG is tailored to address the issue of sparse supervision information. In SAM-PAG, we adopt the advanced SAM to expand sparse scribbles to dense pixel-level pseudo annotations through the dual-branch structure and the consistency of segmentation masks. In $S^2$Diff, we adopt the diffusion model to iteratively refine the noisy saliency maps with the guidance of conditional information, generating accurate saliency maps. Naturally, the core of our $S^2$Diff lies in the acquisition of conditional features and the denoising of saliency maps. For the former, we employ a cross-modal conditional generation module to interweave cross-modal features through frequency integration and implicit-explicit state space interaction, effectively achieving global conditional features. For the latter, we employ a context injection module to mitigate noise interference and to enhance object information with the conditional context. With the close cooperation of SAM-PAG and $S^2$Diff, our method outperforms relevant scribble-supervised methods and achieves competitive performance compared to fully-supervised methods on seven datasets. The code and results of our method are available at https://github.com/Switch457/WeakS2Diff_SOD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。