用语义引导的扩散模型,一键修复高压缩率JPEG图像的模糊纹理。
SODiff: Semantic-Oriented Diffusion Model for JPEG Compression Artifacts Removal
- 通过语义对齐提示提取器,将低质图像特征映射到与文本编码器一致的语义空间。
- 在Real-World JPEG数据集上,峰值信噪比提升1.23dB,结构相似性提高0.018。
- 适合需要快速高保真恢复图像细节的图像编辑、数字存档场景。
JPEG作为广泛使用的图像压缩标准,在高压缩比下常引入严重视觉伪影。尽管现有深度学习恢复方法取得显著进展,仍难以还原复杂纹理细节,导致输出过度平滑。为此,我们提出SODiff——一种新型高效的单步语义导向扩散模型,用于去除JPEG伪影。核心思想是通过语义引导增强预训练扩散模型的生成先验能力。SODiff引入语义对齐图像提示提取器(SAIPE),从低质量图像中提取丰富特征,并将其投影至与文本编码器语义对齐的嵌入空间,同时保留重建所需关键信息。此外,提出质量因子感知时间预测器,隐式学习低质图像的压缩质量因子(QF),并自适应选择最优去噪起始时间步。大量实验表明,SODiff在视觉质量与定量指标上均优于近期领先方法。代码已公开:https://github.com/frakenation/SODiff。
原文摘要 · Abstract (English)
JPEG, as a widely used image compression standard, often introduces severe visual artifacts when achieving high compression ratios. Although existing deep learning-based restoration methods have made considerable progress, they often struggle to recover complex texture details, resulting in over-smoothed outputs. To overcome these limitations, we propose SODiff, a novel and efficient semantic-oriented one-step diffusion model for JPEG artifacts removal. Our core idea is that effective restoration hinges on providing semantic-oriented guidance to the pre-trained diffusion model, thereby fully leveraging its powerful generative prior. To this end, SODiff incorporates a semantic-aligned image prompt extractor (SAIPE). SAIPE extracts rich features from low-quality (LQ) images and projects them into an embedding space semantically aligned with that of the text encoder. Simultaneously, it preserves crucial information for faithful reconstruction. Furthermore, we propose a quality factor-aware time predictor that implicitly learns the compression quality factor (QF) of the LQ image and adaptively selects the optimal denoising start timestep for the diffusion process. Extensive experimental results show that our SODiff outperforms recent leading methods in both visual quality and quantitative metrics. Code is available at: https://github.com/frakenation/SODiff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。