RTSR实时提升AV1压缩视频画质，360p变1080p、540p变4K
RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content
- 用双教师知识蒸馏优化CNN，专为AV1编码视频设计
- 在低复杂度下实现最佳画质与效率平衡，获竞赛第一
- 适合移动端实时视频增强，代码将开源
超分辨率（SR）是通过提升空间分辨率并重建细节来改善视频视觉质量的关键技术。在视频流媒体中，通常传输压缩的低分辨率内容，再通过超分辨率技术还原为更高清画面。为支持实时播放，需兼顾快速推理与高质量重建，但现有基于复杂深度神经网络的方法难以兼顾。本文提出轻量级超分辨率方法RTSR，针对AV1（SVT）编码内容，在不同量化等级下优化，实现从360p到1080p、540p到4K的分辨率提升。该方法采用基于CNN的网络架构，结合双教师知识蒸馏策略。该方案提交至AIM 2024视频超分辨率挑战赛，专攻高效/移动实时视频超分辨率赛道，最终在PSNR、SSIM和VMAF指标上取得最优复杂度-性能权衡，六项参赛方案中排名第一。代码即将开源。
原文摘要 · Abstract (English)
Super-resolution (SR) is a key technique for improving the visual quality of video content by increasing its spatial resolution while reconstructing fine details. SR has been employed in many applications including video streaming, where compressed low-resolution content is typically transmitted to end users and then reconstructed with a higher resolution and enhanced quality. To support real-time playback, it is important to implement fast SR models while preserving reconstruction quality; however most existing solutions, in particular those based on complex deep neural networks, fail to do so. To address this issue, this paper proposes a low-complexity SR method, RTSR, designed to enhance the visual quality of compressed video content, focusing on resolution up-scaling from a) 360p to 1080p and from b) 540p to 4K. The proposed approach utilizes a CNN-based network architecture, which was optimized for AV1 (SVT)-encoded content at various quantization levels based on a dual-teacher knowledge distillation method. This method was submitted to the AIM 2024 Video Super-Resolution Challenge, specifically targeting the Efficient/Mobile Real-Time Video Super-Resolution competition. It achieved the best trade-off between complexity and coding performance (measured in PSNR, SSIM and VMAF) among all six submissions. The code will be available soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。