用生成模型提升多帧原始图像的超分辨率，解决相机流水线中的细节丢失问题。
GenMFSR: Generative Multi-Frame Image Restoration and Super-Resolution
- 基于基础模型先验，从多帧原始图像中恢复亚像素级细节
- 设计高频区域约束损失，避免低频伪影，提升重建质量
- 首个端到端的原始域多帧超分辨率框架，适用于手机相机流水线
相机流水线接收原始Bayer格式帧，需去噪、去马赛克并常进行超分辨率处理。通过捕捉多帧利用自然手抖来提升分辨率，多帧超分辨率因此成为相机流水线的核心问题。现有对抗性方法受限于真实标签质量。我们提出GenMFSR，首个生成式多帧原始图像到RGB的超分辨率流水线，融合基础模型图像先验以获取亚像素信息，适用于相机ISP应用。GenMFSR可对齐多帧原始图像，不同于现有单帧超分辨率方法，并提出一种损失项，限制生成仅在原始域的高频区域进行，从而防止低频伪影。
原文摘要 · Abstract (English)
Camera pipelines receive raw Bayer-format frames that need to be denoised, demosaiced, and often super-resolved. Multiple frames are captured to utilize natural hand tremors and enhance resolution. Multi-frame super-resolution is therefore a fundamental problem in camera pipelines. Existing adversarial methods are constrained by the quality of ground truth. We propose GenMFSR, the first Generative Multi-Frame Raw-to-RGB Super Resolution pipeline, that incorporates image priors from foundation models to obtain sub-pixel information for camera ISP applications. GenMFSR can align multiple raw frames, unlike existing single-frame super-resolution methods, and we propose a loss term that restricts generation to high-frequency regions in the raw domain, thus preventing low-frequency artifacts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。