arXiv:2602.00198eess.IVcs.AI2026-02被引 2

用真实压缩误差生成代理梯度,实现端到端视频自适应码率训练

SCALED : Surrogate-gradient for Codec-Aware Learning of Downsampling in ABR Streaming

  • 通过实际压缩误差构建数据驱动的代理梯度,使非可微编码器可训练
  • 在多下采样比例下,BD-BR(PSNR)提升5.19%,覆盖完整率失真范围
  • 适合需要真实部署性能对齐的自适应码率流媒体系统优化

视频消费的快速增长给现代流媒体架构带来了显著挑战。目前主流的过顶网(OTT)视频分发依赖自适应码率(ABR)流媒体技术,根据客户端的显示能力和网络带宽动态调整码率和分辨率。该流程通常包括对原始高分辨率内容进行下采样、编码传输,再在客户端解码并上采样。传统上各处理阶段独立优化,导致端到端率失真(R-D)性能不优。深度学习的发展推动了对整个ABR流程联合优化的研究,但端到端训练仍因标准视频编码器不可微而困难。现有方法采用可微代理模型(如神经网络或含软量化等可微组件的混合编码方案)近似编码器行为。尽管如此,这些代理模型仍是近似,可能无法完全捕捉标准编码器的真实特性。据我们所知,此前尚无研究证明在训练中使用标准编码器的效率低下。本文提出一种新框架,利用实际压缩误差生成的数据驱动代理梯度,实现了与真实非可微编码器的端到端训练,提升了训练目标与实际部署性能的一致性。实验结果表明,在多个下采样比例下,相比编码器无关训练方法,平均提升5.19%的BD-BR(PSNR),覆盖完整的率失真凸包。

原文摘要 · Abstract (English)

The rapid growth in video consumption has introduced significant challenges to modern streaming architectures. Over-the-Top (OTT) video delivery now predominantly relies on Adaptive Bitrate (ABR) streaming, which dynamically adjusts bitrate and resolution based on client-side constraints such as display capabilities and network bandwidth. This pipeline typically involves downsampling the original high-resolution content, encoding and transmitting it, followed by decoding and upsampling on the client side. Traditionally, these processing stages have been optimized in isolation, leading to suboptimal end-to-end rate-distortion (R-D) performance. The advent of deep learning has spurred interest in jointly optimizing the ABR pipeline using learned resampling methods. However, training such systems end-to-end remains challenging due to the non-differentiable nature of standard video codecs, which obstructs gradient-based optimization. Recent works have addressed this issue using differentiable proxy models, based either on deep neural networks or hybrid coding schemes with differentiable components such as soft quantization, to approximate the codec behavior. While differentiable proxy codecs have enabled progress in compression-aware learning, they remain approximations that may not fully capture the behavior of standard, non-differentiable codecs. To our knowledge, there is no prior evidence demonstrating the inefficiencies of using standard codecs during training. In this work, we introduce a novel framework that enables end-to-end training with real, non-differentiable codecs by leveraging data-driven surrogate gradients derived from actual compression errors. It facilitates the alignment between training objectives and deployment performance. Experimental results show a 5.19\% improvement in BD-BR (PSNR) compared to codec-agnostic training approaches, consistently across the entire rate-distortion convex hull spanning multiple downsampling ratios.

ABR流媒体端到端优化代理梯度视频编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。