arXiv:2502.20824cs.CVeess.IV2025-02CVPR被引 3

用合成数据训练的GAN模型,提升手机拍摄多帧图像的超分辨率效果。

MFSR-GAN: Multi-Frame Super-Resolution with Handheld Motion Modeling

  • 基于多曝光静态图生成含真实噪声和运动的训练数据
  • 在真实手持拍摄数据上实现更清晰、更自然的超分结果
  • 适合手机影像增强、低光环境图像重建场景

智能手机相机虽普及,但小尺寸传感器与紧凑光学系统常导致空间分辨率低并引入畸变。通过融合多帧低分辨率(LR)图像以生成高分辨率(HR)图像,可缓解此类限制。然而现有方法受限于缺乏真实手持连拍中典型噪声与运动模式的数据集。本文提出一种新型合成数据引擎,利用多曝光静态图像生成保留传感器特有噪声及手持拍摄运动特征的LR-HR训练对。同时提出MFSR-GAN:一种多尺度RAW-to-RGB网络用于多帧超分辨率。相比先前方法,MFSR-GAN在架构中强调“基准帧”以减少伪影。在合成与真实数据上的实验表明,使用该合成引擎训练的MFSR-GAN在真实世界多帧超分辨率任务中,重建图像更锐利、更逼真。

原文摘要 · Abstract (English)

Smartphone cameras have become ubiquitous imaging tools, yet their small sensors and compact optics often limit spatial resolution and introduce distortions. Combining information from multiple low-resolution (LR) frames to produce a high-resolution (HR) image has been explored to overcome the inherent limitations of smartphone cameras. Despite the promise of multi-frame super-resolution (MFSR), current approaches are hindered by datasets that fail to capture the characteristic noise and motion patterns found in real-world handheld burst images. In this work, we address this gap by introducing a novel synthetic data engine that uses multi-exposure static images to synthesize LR-HR training pairs while preserving sensor-specific noise characteristics and image motion found during handheld burst photography. We also propose MFSR-GAN: a multi-scale RAW-to-RGB network for MFSR. Compared to prior approaches, MFSR-GAN emphasizes a "base frame" throughout its architecture to mitigate artifacts. Experimental results on both synthetic and real data demonstrates that MFSR-GAN trained with our synthetic engine yields sharper, more realistic reconstructions than existing methods for real-world MFSR.

超分辨率手机摄影GAN多帧重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。