针对退化图像的交互分割,提出基于提示引导的特征增强方法。
PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation

- 用用户提示和先前掩码引导特征恢复区域,聚焦重要部位。
- 在多尺度特征融合与前景重建损失下,提升细节恢复能力。
- 适用于医学影像等退化严重的交互分割场景。
段落自适应模型(SAM)在可提示图像分割中展现出强大的零样本泛化能力,但在实际成像伪影(如噪声、模糊、压缩)下性能显著下降。现有方法全局恢复特征,忽略分割相关区域,且未考虑SAM的迭代优化机制,导致交互设置下表现不佳。本文提出提示引导特征增强的SAM(PGE-SAM),通过提示引导生成器,利用用户提示和先前掩码预测,空间上引导特征修复至关注区域。为恢复退化中丢失的细粒度信息,引入多尺度特征交互,融合低层编码器特征,并设计前景重建损失,将特征级监督限制在分割目标内。此外,构建了DM-Seg基准,涵盖多种医学影像模态,在不同严重程度下包含通用与模态特异性退化。大量实验表明,PGE-SAM在医疗与自然图像域中均达到最先进鲁棒性,跨多个退化等级表现优异,同时保持对干净图像的泛化能力,参数增量不足先前方法的五分之一。
原文摘要 · Abstract (English)
Segment Anything Model (SAM) has revolutionized promptable image segmentation with strong zero-shot generalization. However, its performance degrades substantially under real-world imaging artifacts such as noise, blur, and compression. Existing methods restore features globally without focusing on segmentation-relevant regions and neglect SAM's iterative refinement mechanism, leading to suboptimal performance in interactive settings. We propose Prompt-Guided Feature Enhancement SAM (PGE-SAM), a framework that explicitly leverages user prompts and prior mask predictions to spatially guide the feature restoration process toward regions of interest through a Prompt Guidance Generator. To recover fine-grained details lost under degradation, we introduce Multi-Scale Features Interaction to incorporate low-level encoder features, along with a Foreground Reconstruction Loss that restricts feature-level supervision to the segmentation target. Furthermore, we present DM-Seg, a benchmark for interactive segmentation on degraded medical images, spanning multiple imaging modalities with both general and modality-specific degradations at varying severity levels. Extensive experiments demonstrate that PGE-SAM achieves SOTA robustness on both medical and natural image domains across multiple degradation levels, while maintaining generalization to clean images and adding less than one-fifth of the parameters of prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。