让分割模型在各种画质下都稳定工作,尤其擅长处理模糊、噪点等低质量图像。
Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement
- 通过生成式潜在空间增强提升模型对低质图像的适应能力
- 在复杂退化条件下分割准确率显著提升,同时保持对清晰图像的泛化性能
- 适合需要高鲁棒性图像分割的工业应用和真实场景
Segment Anything Models(SAM)虽在零样本分割上表现优异,但在严重退化的低质量图像上性能急剧下降,限制了其在真实场景中的应用。为此,我们提出GleSAM++,利用生成式潜在空间增强来提升对低质图像的鲁棒性,实现跨图像质量的泛化。为改善预训练扩散模型与分割框架的兼容性,引入特征分布对齐(FDA)和通道复制扩展(CRE)。然而,现有方法缺乏对退化程度的显式指导,模型需隐式拟合从轻度噪声到严重伪影的复杂噪声分布,增加学习负担并导致重建不佳。为此,我们进一步提出退化感知自适应增强(DAE)机制,将任意质量特征的重建分解为退化等级预测与退化感知重建两个阶段。该方法仅需少量可学习参数即可应用于预训练SAM和SAM2,实现高效优化。大量实验证明,GleSAM++在复杂退化条件下显著提升分割鲁棒性,同时保持对清晰图像的泛化能力,并在未见退化类型上表现良好,验证了方法的通用性与数据集的有效性。
原文摘要 · Abstract (English)
Segment Anything Models (SAMs), known for their exceptional zero-shot segmentation performance, have garnered significant attention in the research community. Nevertheless, their performance drops significantly on severely degraded, low-quality images, limiting their effectiveness in real-world scenarios. To address this, we propose GleSAM++, which utilizes Generative Latent space Enhancement to boost robustness on low-quality images, thus enabling generalization across various image qualities. Additionally, to improve compatibility between the pre-trained diffusion model and the segmentation framework, we introduce two techniques, i.e., Feature Distribution Alignment (FDA) and Channel Replication and Expansion (CRE). However, the above components lack explicit guidance regarding the degree of degradation. The model is forced to implicitly fit a complex noise distribution that spans conditions from mild noise to severe artifacts, which substantially increases the learning burden and leads to suboptimal reconstructions. To address this issue, we further introduce a Degradation-aware Adaptive Enhancement (DAE) mechanism. The key principle of DAE is to decouple the reconstruction process for arbitrary-quality features into two stages: degradation-level prediction and degradation-aware reconstruction. Our method can be applied to pre-trained SAM and SAM2 with only minimal additional learnable parameters, allowing for efficient optimization. Extensive experiments demonstrate that GleSAM++ significantly improves segmentation robustness on complex degradations while maintaining generalization to clear images. Furthermore, GleSAM++ also performs well on unseen degradations, underscoring the versatility of our approach and dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。