针对真实退化图像的超分辨率，提出感知退化条件嵌入新方法。
DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution
- 设计退化选择策略与对比学习结合的实时嵌入提取器（REE）
- 在Real-World SR数据集上提升识别准确率，重建图像更清晰自然
- 适合关注真实场景图像恢复的研究者和开发者
多模态大模型虽能利用语言类别作为条件信息解决真实世界图像超分辨率问题，但在退化图像上的表现仍有限。本文重新评估了Recognize Anything Model（RAM）在退化图像上的能力，发现直接在退化空间进行对比学习微调难以取得理想效果。为此，我们提出一种退化选择策略，构建实时嵌入提取器（REE），通过对比学习显著提升退化图像内容的识别性能。进一步地，采用条件特征调制器（CFM）将REE的高层语义信息融入基于Mamba的网络，有效利用像素级信息恢复纹理，生成视觉上更令人满意的图像。大量实验表明，该方法可使超分辨率网络在保真度与感知质量间取得更好平衡，凸显Mamba在真实应用中的潜力。代码已开源：https://github.com/nathan66666/DACESR.git
原文摘要 · Abstract (English)
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。