用B样条和傅里叶映射提升视频超分的时空细节与连贯性。
BF-STVSR: B-Splines and Fourier-Best Friends for High Fidelity Spatial-Temporal Video Super-Resolution
- 用B样条实现平滑时序插值，提升运动连续性
- 通过傅里叶映射捕捉主导空间频率，增强纹理细节
- 不依赖预训练光流，避免性能退化，适合高保真视频重建
现有连续时空视频超分辨率（C-STVSR）方法多采用隐式神经表示（INR）进行连续编码，但常因仅依赖坐标拼接和预训练光流网络来表征运动，难以捕捉视频数据复杂性。有趣的是，我们发现加入位置编码不仅未提升性能，反而导致下降，尤其在结合预训练光流时更限制模型灵活性。为此，提出BF-STVSR框架，包含两个关键模块：1）用于平滑时序插值的B样条映射器；2）用于捕捉主导空间频率的傅里叶映射器。该方法在多种指标（如PSNR、SSIM）上达到当前最优，显著提升空间细节与时间一致性。代码已开源。
原文摘要 · Abstract (English)
While prior methods in Continuous Spatial-Temporal Video Super-Resolution (C-STVSR) employ Implicit Neural Representation (INR) for continuous encoding, they often struggle to capture the complexity of video data, relying on simple coordinate concatenation and pre-trained optical flow networks for motion representation. Interestingly, we find that adding position encoding, contrary to common observations, does not improve--and even degrades--performance. This issue becomes particularly pronounced when combined with pre-trained optical flow networks, which can limit the model's flexibility. To address these issues, we propose BF-STVSR, a C-STVSR framework with two key modules tailored to better represent spatial and temporal characteristics of video: 1) B-spline Mapper for smooth temporal interpolation, and 2) Fourier Mapper for capturing dominant spatial frequencies. Our approach achieves state-of-the-art in various metrics, including PSNR and SSIM, showing enhanced spatial details and natural temporal consistency. Our code is available https://github.com/Eunjnnn/bfstvsr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。