arXiv:2411.10798eess.IVcs.CV2024-11被引 27

利用手机拍摄的RAW数据恢复真实图像细节,提升超分辨率质量。

Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution

  • 用RAW数据补足RGB图像丢失的细节信息
  • 在10个评估指标上显著提升超分辨率性能
  • 适合做真实场景图像增强的研究者使用

真实世界图像超分辨率(Real SR)旨在从低分辨率(LR)图像生成高保真、细节丰富的高分辨率(HR)图像。现有方法主要依赖LR RGB域生成细节,常导致细节贫乏或失真。本文首次提出利用图像信号处理中被丢失的RAW数据来补充现有仅基于RGB的方法,实现更优输出。我们指出,去噪和去马赛克等关键图像处理步骤会损失细节,使LR RAW成为有价值的信息源。为此,构建了包含超过10,000对样本的RealSR-RAW数据集,涵盖多款手机、不同焦距与多样场景下的LR/HR RGB及对应LR RAW。同时提出一种通用的RAW适配器,可高效融合至CNN、Transformer和扩散模型中,通过抑制噪声并校准分布实现特征对齐。大量实验表明,引入RAW数据显著增强细节恢复能力,在10项评估指标上均有提升,涵盖保真度与感知质量。研究成果为真实超分辨率开辟新方向,数据集与代码将公开以支持后续研究。

原文摘要 · Abstract (English)

Real-world image super-resolution (Real SR) aims to generate high-fidelity, detail-rich high-resolution (HR) images from low-resolution (LR) counterparts. Existing Real SR methods primarily focus on generating details from the LR RGB domain, often leading to a lack of richness or fidelity in fine details. In this paper, we pioneer the use of details hidden in RAW data to complement existing RGB-only methods, yielding superior outputs. We argue that key image processing steps in Image Signal Processing, such as denoising and demosaicing, inherently result in the loss of fine details in LR images, making LR RAW a valuable information source. To validate this, we present RealSR-RAW, a comprehensive dataset comprising over 10,000 pairs with LR and HR RGB images, along with corresponding LR RAW, captured across multiple smartphones under varying focal lengths and diverse scenes. Additionally, we propose a novel, general RAW adapter to efficiently integrate LR RAW data into existing CNNs, Transformers, and Diffusion-based Real SR models by suppressing the noise contained in LR RAW and aligning its distribution. Extensive experiments demonstrate that incorporating RAW data significantly enhances detail recovery and improves Real SR performance across ten evaluation metrics, including both fidelity and perception-oriented metrics. Our findings open a new direction for the Real SR task, with the dataset and code will be made available to support future research.

超分辨率RAW数据图像增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。