用事件信号提升模糊视频超分辨率,恢复更清晰动态画面。
Event-Enhanced Blurry Video Super-Resolution
- 引入事件信号与帧图像互馈,增强去模糊特征表达
- 在真实数据上比现有最佳方法高2.59 dB,快7.28倍
- 适合需要高速高精度视频重建的场景,如自动驾驶
本文针对模糊视频超分辨率(BVSR)任务,旨在从低分辨率且模糊的输入中生成高分辨率视频。现有方法因运动信息不足和低分辨率帧高频细节缺失,难以恢复清晰细节,导致明显伪影和抖动。为此,我们引入事件信号,提出新型事件增强网络Ev-DeblurVSR。为有效融合帧与事件信息进行特征去模糊,设计了双向特征去模糊模块,利用帧内事件的运动信息去模糊帧特征,同时以帧的全局场景上下文增强事件特征。为进一步提升时序一致性,提出混合可变形对齐模块,充分融合帧间事件与光流的互补运动信息,改进可变形对齐中的运动估计。大量实验表明,Ev-DeblurVSR在合成与真实数据集上均达到新基准性能。尤其在真实数据上,相比近期最优基线FMA-Net,准确率提升2.59 dB,推理速度加快7.28倍。代码已开源:https://github.com/DachunKai/Ev-DeblurVSR。
原文摘要 · Abstract (English)
In this paper, we tackle the task of blurry video super-resolution (BVSR), aiming to generate high-resolution (HR) videos from low-resolution (LR) and blurry inputs. Current BVSR methods often fail to restore sharp details at high resolutions, resulting in noticeable artifacts and jitter due to insufficient motion information for deconvolution and the lack of high-frequency details in LR frames. To address these challenges, we introduce event signals into BVSR and propose a novel event-enhanced network, Ev-DeblurVSR. To effectively fuse information from frames and events for feature deblurring, we introduce a reciprocal feature deblurring module that leverages motion information from intra-frame events to deblur frame features while reciprocally using global scene context from the frames to enhance event features. Furthermore, to enhance temporal consistency, we propose a hybrid deformable alignment module that fully exploits the complementary motion information from inter-frame events and optical flow to improve motion estimation in the deformable alignment process. Extensive evaluations demonstrate that Ev-DeblurVSR establishes a new state-of-the-art performance on both synthetic and real-world datasets. Notably, on real data, our method is +2.59 dB more accurate and 7.28$\times$ faster than the recent best BVSR baseline FMA-Net. Code: https://github.com/DachunKai/Ev-DeblurVSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。