一次处理模糊、低清、低帧率,让视频瞬间变清晰流畅
Joint Video Enhancement with Deblurring, Super-Resolution, and Frame Interpolation Network
- 设计联合网络同时完成去模糊、超分辨率和插帧
- 在公开数据集上性能超越串联式方法,模型更小速度更快
- 适合需要实时高清视频修复的场景,如直播与监控
视频质量常受多种因素共同影响,而非单一问题。传统方法通过依次应用去模糊、超分辨率和帧插值等技术进行修复,但效率低且效果不佳,因各模块未考虑多退化因素共存的情况。本文提出一种联合视频增强方法,通过构建统一优化问题,一次性恢复高分辨率、高帧率且清晰的视频。所提网络名为DSFN,其包含联合去模糊与超分辨率(JDSR)模块,用于提升输入低分辨率模糊帧;以及基于三帧的帧插值(TFBFI)模块,生成相邻帧间的中间帧。二者协同工作,在不增加复杂度的前提下实现优异性能。实验表明,该方法在多个公开数据集上优于现有串行方法,且模型更小、处理更快。
原文摘要 · Abstract (English)
Video quality is often severely degraded by multiple factors rather than a single factor. These low-quality videos can be restored to high-quality videos by sequentially performing appropriate video enhancement techniques. However, the sequential approach was inefficient and sub-optimal because most video enhancement approaches were designed without taking into account that multiple factors together degrade video quality. In this paper, we propose a new joint video enhancement method that mitigates multiple degradation factors simultaneously by resolving an integrated enhancement problem. Our proposed network, named DSFN, directly produces a high-resolution, high-frame-rate, and clear video from a low-resolution, low-frame-rate, and blurry video. In the DSFN, low-resolution and blurry input frames are enhanced by a joint deblurring and super-resolution (JDSR) module. Meanwhile, intermediate frames between input adjacent frames are interpolated by a triple-frame-based frame interpolation (TFBFI) module. The proper combination of the proposed modules of DSFN can achieve superior performance on the joint video enhancement task. Experimental results show that the proposed method outperforms other sequential state-of-the-art techniques on public datasets with a smaller network size and faster processing time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。