arXiv:2504.02216eess.IVcs.CV2025-04被引 3

让图像压缩兼顾机器视觉效果,显著降低码率。

Image Coding for Machines via Feature-Preserving Rate-Distortion Optimization

  • 用特征距离替代传统误差,优化压缩对下游任务的影响。
  • 相比仅保视觉质量的压缩,任务准确率相同时可节省17%码率。
  • 适合需要高效图像传输的计算机视觉系统,如自动驾驶、监控。

大量图像和视频主要由计算机视觉算法处理,人类仅偶尔查看。在分布式应用中需压缩前处理时,编码方法必须兼顾视觉质量与下游任务性能。我们理论上证明:通过以原始图像与解码图像特征间的距离作为失真度量进行率失真优化(RDO),可减轻压缩对任务性能的影响。但直接优化该目标计算成本过高,需反复编码解码并评估特征。为此,我们简化RDO公式,使失真项可通过块编码方式计算。首先对特征提取器进行泰勒展开,将特征距离转化为包含神经网络雅可比矩阵的二次度量;再引入块级近似,称为输入相关平方误差(IDSE);最后用雅可比矩阵的草图近似该度量。最终损失可在变换域逐块计算,并与平方误差(SSE)结合,同时优化视觉质量与计算机视觉性能。在AVC和HEVC上对多个特征提取器与下游网络的仿真表明,相同任务精度下,相比基于SSE的RDO,最多可节省17%码率,解码复杂度无增加,编码复杂度仅上升7.86%。

原文摘要 · Abstract (English)

Many images and videos are primarily processed by computer vision algorithms, involving only occasional human inspection. When this content requires compression before processing, e.g., in distributed applications, coding methods must optimize for both visual quality and downstream task performance. We first show theoretically that an approach to reduce the effect of compression for a given task loss is to perform rate-distortion optimization (RDO) using the distance between features, obtained from the original and the decoded images, as a distortion metric. However, optimizing directly such a rate-distortion objective is computationally impractical because it requires iteratively encoding and decoding the entire image-plus feature evaluation-for each possible coding configuration. We address this problem by simplifying the RDO formulation to make the distortion term computable using block-based encoders. We first apply Taylor's expansion to the feature extractor, recasting the feature distance as a quadratic metric involving the Jacobian matrix of the neural network. Then, we replace the linearized metric with a block-wise approximation, which we call input-dependent squared error (IDSE). To make the metric computable, we approximate IDSE using sketches of the Jacobian. The resulting loss can be evaluated block-wise in the transform domain and combined with the sum of squared errors (SSE) to address both visual quality and computer vision performance. Simulations with AVC and HEVC across multiple feature extractors and downstream networks show up to 17 % bit-rate savings for the same task accuracy compared to RDO based on SSE, with no decoder complexity overhead and a small (7.86 %) encoder complexity increase.

图像压缩机器视觉率失真优化特征保持

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。