用强化学习动态生成视频码率阶梯，兼顾画质与解码效率。
DQ-Ladder: A Deep Reinforcement Learning-based Bitrate Ladder for Adaptive Video Streaming
- 基于DQN的强化学习框架，根据预测解码时间与画质自动构建码率阶梯。
- 相比HLS基准，码率降低10.3%以上，解码时间减少22%。
- 对预测误差鲁棒性强，噪声达20%仍稳定运行，适合实际部署。
自适应视频流通常依赖预定义的码率-分辨率组合（码率阶梯）。但固定阶梯常忽略内容差异与解码复杂性，导致编码时间、解码效率与画质之间的权衡不佳。本文提出DQ-Ladder，一种基于深度强化学习（DRL）的时序与质量感知码率阶梯构建方案。DQ-Ladder以每段视频的预测解码时间、质量评分和码率水平为输入，通过加权奖励函数（包含解码时间、视频质量、分辨率平滑度）指导深度Q网络（DQN）代理。我们利用机器学习模型预测解码时间、码率水平及客观质量指标（VMAF、XPSNR），避免了耗时的编码或质量评估。在750个视频序列上，使用基于苹果HLS兼容的六种分辨率和41个量化参数的通用视频编码（VVC）工具链（VVenC/VVdeC）进行评估。实验结果表明，相比四种基线方法，DQ-Ladder在XPSNR上实现至少10.3%的BD-rate降低，同时解码时间减少22%。此外，其对预测误差敏感度显著低于现有方法，在高达20%噪声下仍保持鲁棒性。
原文摘要 · Abstract (English)
Adaptive streaming of segmented video over HTTP typically relies on a predefined set of bitrate-resolution pairs, known as a bitrate ladder. However, fixed ladders often overlook variations in content and decoding complexities, leading to suboptimal trade-offs between encoding time, decoding efficiency, and video quality. This article introduces DQ-Ladder, a deep reinforcement learning (DRL)-based scheme for constructing time- and quality-aware bitrate ladders for adaptive video streaming applications. DQ-Ladder employs predicted decoding time, quality scores, and bitrate levels per segment as inputs to a Deep Q-Network (DQN) agent, guided by a weighted reward function of decoding time, video quality, and resolution smoothness. We leverage machine learning models to predict decoding time, bitrate level, and objective quality metrics (VMAF, XPSNR), eliminating the need for exhaustive encoding or quality metric computation. We evaluate DQ-Ladder using the Versatile Video Coding (VVC) toolchain (VVenC/VVdeC) on 750 video sequences across six Apple HLS-compliant resolutions and 41 quantization parameters. Experimental results against four baselines show that DQ-Ladder achieves BD-rate reductions of at least 10.3% for XPSNR compared to the HLS ladder, while reducing decoding time by 22%. DQ-Ladder shows significantly lower sensitivity to prediction errors than competing methods, remaining robust even with up to 20% noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。