提出轻量级分类器,实时自适应选择视频编码的下采样比例。
Fast-OMRA: Fast Online Motion Resolution Adaptation for Neural B-Frame Coding
- 通过观察参考帧和编码状态,判断是否需要下采样
- 两种分类器分别用二分类和软标签训练,效果接近暴力搜索
- 无需重训练原有编码器,适合实时视频压缩场景
采用分层时间预测的神经B帧编码器常因训练与测试时的图像组(GOP)大小不一致,导致运动估计网络无法准确预测大运动。现有缓解方法需对视频帧下采样以提升运动估计性能,但最优下采样因子需通过耗时的率失真优化确定。本文提出轻量级分类器,仅依赖编码状态与参考帧信息,实时预测最佳下采样因子。设计了二分类与多分类两种变体:前者使用焦点损失训练,区分高/低分辨率运动估计;后者采用新型软标签,融合不同下采样因子的率失真代价知识进行训练。两类分类器作为附加模块运行,无需重新训练原编码器。实验表明,其编码性能可媲美暴力搜索方法,同时大幅降低计算复杂度。
原文摘要 · Abstract (English)
Most learned B-frame codecs with hierarchical temporal prediction suffer from the domain shift issue caused by the discrepancy in the Group-of-Pictures (GOP) size used for training and test. As such, the motion estimation network may fail to predict large motion properly. One effective strategy to mitigate this domain shift issue is to downsample video frames for motion estimation. However, finding the optimal downsampling factor involves a time-consuming rate-distortion optimization process. This work introduces lightweight classifiers to determine the downsampling factor. To strike a good rate-distortion-complexity trade-off, our classifiers observe simple state signals, including only the coding and reference frames, to predict the best downsampling factor. We present two variants that adopt binary and multi-class classifiers, respectively. The binary classifier adopts the Focal Loss for training, classifying between motion estimation at high and low resolutions. Our multi-class classifier is trained with novel soft labels incorporating the knowledge of the rate-distortion costs of different downsampling factors. Both variants operate as add-on modules without the need to re-train the B-frame codec. Experimental results confirm that they achieve comparable coding performance to the brute-force search methods while greatly reducing computational complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。