arXiv:2501.12216cs.LGcs.CV2025-01CVPR被引 5

用强化学习优化视频编码块级量化参数,提升下游任务性能。

RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression

  • 通过块级量化参数控制,实现任务相关区域优先编码。
  • 在相同码率下,目标检测和显著区域编码任务性能显著提升。
  • 推理时无需输入下游任务,适合车载等实时场景。

视频编码器通常在码率约束下最小化重建误差以优化人眼感知质量。但在自动驾驶等现代应用中,视频主要作为人工智能系统(如目标识别、分割)的输入,而非供人观看。因此,为下游任务优化编码更具意义。然而,如何将此类优化与现有高效且广泛使用的标准视频编码器结合仍是挑战。本文提出在宏块级别控制量化参数(QPs),实现对帧内任务相关区域的精细化编码。将该优化问题建模为强化学习任务,智能体学习权衡选择不同量化参数对任务性能和码率的长期影响。值得注意的是,推理阶段无需提供下游任务信息,适用于流媒体和车载边缘设备。实验表明,在车辆检测和感兴趣区域(ROI)编码两个任务中,本方法在相同码率下显著优于传统无任务感知编码方法,为更高效的任务感知视频压缩提供了新路径。

原文摘要 · Abstract (English)

Video encoders optimize compression for human perception by minimizing reconstruction error under bit-rate constraints. In many modern applications such as autonomous driving, an overwhelming majority of videos serve as input for AI systems performing tasks like object recognition or segmentation, rather than being watched by humans. It is therefore useful to optimize the encoder for a downstream task instead of for perceptual image quality. However, a major challenge is how to combine such downstream optimization with existing standard video encoders, which are highly efficient and popular. Here, we address this challenge by controlling the Quantization Parameters (QPs) at the macro-block level to optimize the downstream task. This granular control allows us to prioritize encoding for task-relevant regions within each frame. We formulate this optimization problem as a Reinforcement Learning (RL) task, where the agent learns to balance long-term implications of choosing QPs on both task performance and bit-rate constraints. Notably, our policy does not require the downstream task as an input during inference, making it suitable for streaming applications and edge devices such as vehicles. We demonstrate significant improvements in two tasks, car detection, and ROI (saliency) encoding. Our approach improves task performance for a given bit rate compared to traditional task agnostic encoding methods, paving the way for more efficient task-aware video compression.

视频压缩强化学习任务感知边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。