提出新方法提升扩散模型视频超分,显著改善画质与帧间一致性。
Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features
- 在特征域进行密集对齐和补偿,优于像素域
- 实现35.82%感知质量提升、0.20 dB清晰度增益
- 适合追求高画质与稳定性的视频增强应用
基于扩散模型的视频超分辨率方法虽具备出色感知质量,但存在误差累积、空间伪影及感知质量与保真度间的权衡问题,主要源于帧间对准不准确与补偿不足。本文重新审视扩散模型框架中相邻帧间的对准与补偿机制,发现特征域相比像素域具有更强的空间-时间相关性,更适合信息补偿;而在上采样分辨率下进行光流扭曲更利于保留高频信息,但该优势非单调。为此,提出新型DGAF-VSR方法,包含光学引导扭曲模块(OGWM)以保持对齐特征中的高频细节,以及特征级时序条件模块(FTCM)实现特征域密集引导。在合成与真实数据集上的大量实验表明,DGAF-VSR在关键指标上超越现有方法:感知质量降低35.82% DISTS,保真度提升0.20 dB PSNR,时序一致性改善30.37% tLPIPS。
原文摘要 · Abstract (English)
Diffusion model (DM) based Video Super-Resolution (VSR) approaches achieve impressive perceptual quality. However, they suffer from error accumulation, spatial artifacts, and a trade-off between perceptual quality and fidelity, primarily caused by inaccurate alignment and insufficient compensation between video frames. In this paper, within the DM-based VSR pipeline, we revisit the role of alignment and compensation between adjacent video frames and reveal two crucial observations: (a) the feature domain is better suited than the pixel domain for information compensation due to its stronger spatial and temporal correlations, and (b) warping at an upscaled resolution better preserves high-frequency information, but this benefit is not necessarily monotonic. Therefore, we propose a novel Densely Guided diffusion model with Aligned Features for Video Super-Resolution (DGAF-VSR), with an Optical Guided Warping Module (OGWM) to maintain high-frequency details in the aligned features and a Feature-wise Temporal Condition Module (FTCM) to deliver dense guidance in the feature domain. Extensive experiments on synthetic and real-world datasets demonstrate that DGAF-VSR surpasses state-of-the-art methods in key aspects of VSR, including perceptual quality (35.82\% DISTS reduction), fidelity (0.20 dB PSNR gain), and temporal consistency (30.37\% tLPIPS reduction).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。