无需微调即可提升神经视频编码的运动估计精度。
Boosting Neural Video Codec via Scale-Driven Online Flow Refinement
- 通过多尺度运动信息动态融合修正误差。
- 在DCVC-FM上实现PSNR增益2.84%、MS-SSIM增益4.05%。
- 适合追求低延迟高泛化性的视频编码应用。
尽管当前神经视频编码器（NVC）性能优异，但在训练中未见的复杂运动模式下泛化能力有限。为在不增加在线微调成本的前提下弥补领域差距，本文提出一种无需训练的尺度驱动在线光流精炼方法（SOFR）。SOFR作为即插即用模块，整合粗细尺度的运动信息，并根据补偿精度动态融合，以极低计算开销有效修正运动估计误差。此外，设计了率感知策略，依据码率模式选择不同融合方式，并基于补偿误差进行可靠性检查，确保鲁棒性。在USTC-TD数据集上的大量实验验证了SOFR在多种NVC框架（如DCVC-SDD、DCVC-FM、EHVC）中的有效性与泛化能力。特别地，对DCVC-FM可实现平均2.84%（PSNR）和4.05%（MS-SSIM）的码率节省，编码时间增加可忽略。代码已开源。
原文摘要 · Abstract (English)
Although state-of-the-art neural video codecs (NVCs) have achieved remarkable performance, they suffer from limited generalization when encountering complex motion patterns unseen during training. To bridge this domain gap without the expensive cost of online fine-tuning, we propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method. Serving as a plug-and-play module, SOFR integrates motion information from coarse and fine scales and dynamically fuses them according to warping accuracy, effectively rectifying motion estimation errors with negligible computational overhead. Furthermore, we design a rate-aware strategy that selects different dynamic fusion strategies according to bitrate modes, and employs a reliability check based on warping error to ensure robustness. Extensive experiments on the USTC-TD dataset verify the effectiveness and generalization of SOFR across various NVC frameworks, including DCVC-SDD, DCVC-FM, and EHVC. Notably, it brings an average of 2.84% and 4.05% bitrate savings in terms of PSNR and MS-SSIM, respectively, to DCVC-FM with negligible coding time increase. Our code is available at https://github.com/SunnyMass/SOFR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。