arXiv:2505.07071cs.CV2025-05IJCAI被引 6

用语义分割指导单步图像超分辨率,提升复杂区域细节

Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution

  • 将语义掩码融入采样过程,动态调整噪声和残差传输
  • 在真实与合成数据上显著提升复杂图像的视觉质量与细节恢复
  • 适合需要快速生成高保真图像的场景,如医学影像或遥感

基于扩散模型的图像超分辨率方法已展现出卓越性能。最近进展引入确定性采样过程,将推理步骤从15步减少至单步,大幅提升了现有扩散模型的推理速度。然而,在处理复杂语义区域时,其效率仍受限于单步推理。为此,我们提出SAMSR,一种融合语义分割掩码的语义引导扩散框架。具体地,我们设计了SAM-Noise模块,利用分割掩码对高斯噪声进行精炼,以保留空间与语义特征;同时提出像素级采样策略,根据像素级语义权重动态调节残差传输率与噪声强度,优先处理语义丰富的区域。为增强训练效果,我们还引入语义一致性损失,使预测与真实之间的像素级语义权重保持一致。在真实世界与合成数据集上的大量实验表明,SAMSR显著提升了感知质量与细节恢复能力,尤其在语义复杂的图像中表现突出。代码已开源:https://github.com/Liu-Zihang/SAMSR。

原文摘要 · Abstract (English)

Diffusion-based image super-resolution (SR) methods have demonstrated remarkable performance. Recent advancements have introduced deterministic sampling processes that reduce inference from 15 iterative steps to a single step, thereby significantly improving the inference speed of existing diffusion models. However, their efficiency remains limited when handling complex semantic regions due to the single-step inference. To address this limitation, we propose SAMSR, a semantic-guided diffusion framework that incorporates semantic segmentation masks into the sampling process. Specifically, we introduce the SAM-Noise Module, which refines Gaussian noise using segmentation masks to preserve spatial and semantic features. Furthermore, we develop a pixel-wise sampling strategy that dynamically adjusts the residual transfer rate and noise strength based on pixel-level semantic weights, prioritizing semantically rich regions during the diffusion process. To enhance model training, we also propose a semantic consistency loss, which aligns pixel-wise semantic weights between predictions and ground truth. Extensive experiments on both real-world and synthetic datasets demonstrate that SAMSR significantly improves perceptual quality and detail recovery, particularly in semantically complex images. Our code is released at https://github.com/Liu-Zihang/SAMSR.

图像超分扩散模型语义引导单步生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。