改进机器特征编码的压缩效率,让比特分配更聪明地适应后处理结构。
Wrapper-Aware Rate-Distortion Optimization in Feature Coding for Machines
- 用加权误差替代平方误差,让压缩时考虑后处理会丢掉的信息。
- 在HEVC上实现效果媲美VVC级的最先进方法,性能提升明显。
- 离线训练固定权重,运行时开销极低,适合实际部署。
机器特征编码(FCM)是一种用于分割推理的有损压缩范式。发送端在将神经网络前半部分的输出传给接收端完成推理前进行编码。现有方法在预处理和后处理神经网络之间嵌入传统编解码器(如视频编码器),称为包装器(wrappers)。由于传统编解码器不可导,包装器通过代理编解码器训练,完成后替换为标准编解码器。这些编解码器基于平方误差(SSE)进行率失真优化(RDO),但未考虑后处理包装器,导致编码器浪费比特保存后处理阶段会丢弃的信息。本文提出一种包装器感知的率失真优化(WA-RDO),通过引入包装器感知的加权平方误差调整比特分配。为使WA-RDO适用于实际的FCM系统,我们提出:1)跨图像组复用权重以降低计算量;2)离线训练固定、依赖架构和任务的权重。在MPEG测试条件下,基于HEVC的实现达到与基于VVC的最先进方法相当的性能,有效弥合了编解码器代际差距,且相对于基于SSE-RDO的HEVC,运行时开销极小。
原文摘要 · Abstract (English)
Feature coding for machines (FCM) is a lossy compression paradigm for split-inference. The transmitter encodes the outputs of the first part of a neural network before sending them to the receiver for completing the inference. Practical FCM methods ``sandwich'' a traditional codec between pre- and post-processing neural networks, called wrappers, to make features easier to compress using video codecs. Since traditional codecs are non-differentiable, the wrappers are trained using a proxy codec, which is later replaced by a standard codec after training. These codecs perform rate-distortion optimization (RDO) based on the sum of squared errors (SSE). Because the RDO does not consider the post-processing wrapper, the inner codec can invest bits in preserving information that the post-processing later discards. In this paper, we modify the bit-allocation in the inner codec via a wrapper-aware weighted SSE metric. To make wrapper-aware RDO (WA-RDO) practical for FCM, we propose: 1) temporal reuse of weights across a group of pictures and 2) fixed, architecture- and task-dependent weights trained offline. Under MPEG test conditions, our methods implemented on HEVC match the VVC-based FCM state-of-the-art, effectively bridging a codec generation gap with minimal runtime overhead relative to SSE-RDO HEVC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。