用原图指导修复重选的低质动态照片,提升画面清晰度。
LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion
- 用原高清图作参考,引导修复重选的低质量帧。
- 在快速运动场景下,修复效果显著优于现有方法。
- 适合需要优化动态照片画质的摄影与视频用户。
Live Photo 同时记录一张高质量静态图像和一段短视频,以保留捕捉瞬间的动态。当用户选择其他帧作为关键照片时,这些帧因视频采集流程的图像处理管线(ISP)质量较低,常出现明显画质下降。这一画质差距凸显了专用修复技术的必要性。为此,我们提出 LiveMoments,一种针对重选关键帧的参考引导图像修复框架。该方法采用双分支神经网络:参考分支从原始高质量关键帧提取结构与纹理信息,主分支则利用参考信息恢复重选帧。此外,我们设计统一的运动对齐模块,在潜在空间与图像空间双重实现运动引导的空间对齐。在真实与合成的 Live Photos 数据集上实验表明,相较于现有方案,LiveMoments 在感知质量和保真度上均有显著提升,尤其在快速运动或复杂结构场景中表现更优。代码已开源:https://github.com/OpenVeraTeam/LiveMoments。
原文摘要 · Abstract (English)
Live Photo captures both a high-quality key photo and a short video clip to preserve the precious dynamics around the captured moment. While users may choose alternative frames as the key photo to capture better expressions or timing, these frames often exhibit noticeable quality degradation, as the photo capture ISP pipeline delivers significantly higher image quality than the video pipeline. This quality gap highlights the need for dedicated restoration techniques to enhance the reselected key photo. To this end, we propose LiveMoments, a reference-guided image restoration framework tailored for the reselected key photo in Live Photos. Our method employs a two-branch neural network: a reference branch that extracts structural and textural information from the original high-quality key photo, and a main branch that restores the reselected frame using the guidance provided by the reference branch. Furthermore, we introduce a unified Motion Alignment module that incorporates motion guidance for spatial alignment at both the latent and image levels. Experiments on real and synthetic Live Photos demonstrate that LiveMoments significantly improves perceptual quality and fidelity over existing solutions, especially in scenes with fast motion or complex structures. Our code is available at https://github.com/OpenVeraTeam/LiveMoments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。