提出新模型同时提升视频清晰度和去模糊,还解决曝光差异带来的问题。
FMA-Net++: Motion- and Exposure-Aware Joint Video Super-Resolution and Deblurring
- 用并行堆叠模块扩大时间感知范围,避免传统方法的延迟或局限。
- 在合成数据上训练后,在真实视频中仍保持高画质和快速推理。
- 适合需要高质量视频重建与实时处理的场景,如监控、影视修复。
联合视频超分辨率与去模糊(VSRDB)需兼具高效长时序建模与对帧级曝光时间变化的鲁棒性,后者会改变视频帧间的运动模糊程度。本文提出FMA-Net++,一种非递归、序列级框架,基于分层精炼与双向聚合(HRBA)块构建。通过堆叠HRBA块,该模型可并行处理视频帧,同时分层扩展时间感受野,克服滑动窗口设计的时间感知局限与递归结构的顺序瓶颈。为应对依赖曝光时间的模糊问题,引入曝光时间感知调制(ETM)层,将来自曝光时间感知特征提取器(ETE)的曝光嵌入条件化至HRBA特征,引导曝光感知光流动态滤波模块预测运动与曝光相关的退化核。FMA-Net++将退化学习与恢复解耦:前者预测退化先验,后者利用其进行高效高分辨率重建。为在受控曝光时间变化下评估VSRDB,本文引入REDS-ME(多曝光)与REDS-RE(随机曝光)基准。仅在合成数据上训练的FMA-Net++在上述基准上达到当前最优精度与时间一致性,并在GoPro及挑战性真实视频上表现出强泛化能力,优于近期方法在重建质量与推理速度上的表现。
原文摘要 · Abstract (English)
Joint video super-resolution and deblurring (VSRDB) requires both efficient long-range temporal modeling and robustness to frame-wise exposure-duration variation, which changes the extent of motion blur across video frames. We propose FMA-Net++, a non-recurrent, sequence-level framework built from Hierarchical Refinement with Bidirectional Aggregation (HRBA) blocks. By stacking HRBA blocks, FMA-Net++ processes video frames in parallel while hierarchically expanding the temporal receptive field, avoiding the limited temporal receptive field of sliding-window designs and the sequential bottleneck of recurrent ones. To handle exposure-duration-dependent blur, we introduce an Exposure Time-aware Modulation (ETM) layer that conditions HRBA features on exposure embeddings from an Exposure Time-aware Feature Extractor (ETE). The conditioned features guide an exposure-aware flow-guided dynamic filtering module to predict motion- and exposure-aware degradation kernels. FMA-Net++ decouples degradation learning from restoration: the former predicts degradation priors and the latter exploits them for efficient high-resolution restoration. To evaluate VSRDB under controlled exposure-duration variation, we introduce the REDS-ME (multi-exposure) and REDS-RE (random-exposure) benchmarks. Trained solely on synthetic data, FMA-Net++ achieves state-of-the-art accuracy and temporal consistency on these benchmarks. It further shows strong out-of-distribution performance on GoPro and challenging real-world videos, while outperforming recent methods in both restoration quality and inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。