arXiv:2606.20671cs.CVcs.AI2026-06

揭示神经编解码器梯度的局部近似机制,提升压缩效率。

A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers

论文配图:A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers
图 1 · 摘自论文原文
  • 将代理梯度解释为编解码器的一阶局部近似,增强可解释性。
  • 在多种编码器上实现最高-23.59%的BD-Rate(PSNR)提升。
  • 适用于全链路神经包装,支持多倍下采样等复杂任务。

神经包装器是用于提升传统视频编码器性能的学习型预处理与后处理网络。尽管这些方法能显著提高压缩效率,但训练仍具挑战性,因编码过程中的多重离散决策导致编码器不可微。代理梯度成为端到端学习的有效方案,避免额外模仿网络,且可提升压缩性能。特别是最近提出的SCALED方法,利用真实压缩误差,在训练下采样等预处理器时表现优异。然而,该梯度最初作为重参数化技巧提出,限制了其可解释性。本文表明,该代理梯度可被解释为视频编码器的一阶局部近似,从而揭示其有效性。我们进一步证明其不仅适用于下采样学习,也适用于更具挑战性的完整神经包装(含预/后处理网络)。此外,该方法在不同视频编码器、质量因子和任务间具有良好泛化能力,包括多倍下采样,相较标准重采样基线,在x264上实现最高-23.59%的BD-Rate(PSNR)改善,在VVenC上实现-20.07%的改善。

原文摘要 · Abstract (English)

Neural wrappers are learned pre-and postprocessing networks designed to enhance the performance of conventional video codecs. Although these approaches can significantly improve compression efficiency, training them remains challenging due to the non-differentiability of video codecs, which arises from the multiple discrete decisions involved in the encoding process. Surrogate gradients have recently emerged as an effective solution for enabling end-to-end learning with conventional codecs. They offer two main advantages: they avoid training an additional network to mimic the codec, and they can improve compression performance. In particular, the recently proposed SCALED method, which leverages the true compression error, has shown strong results for training neural pre-processors such as downscalers. However, this SCALED gradient was originally introduced as a reparameterization trick, which limits its interpretability. In this paper, we show that this surrogate gradient can be interpreted as a first-order local approximation of the video codec, providing insight into its effectiveness. We further demonstrate that it is effective not only for learning downscaling operations, but also for the more challenging task of full neural wrapping with pre-and post-processing networks. Finally, we show that the approach generalizes well across different video codecs, quality factors, and tasks, including multiple downscaling ratios, yielding BD-Rate (PSNR) reductions of up to -23.59% on x264 and -20.07% on VVenC relative to standard resampling baselines.

神经编码代理梯度视频压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。