让图像超分模型学会应对后续压缩，提升压缩效率
Versatile Recompression-Aware Perceptual Image Super-Resolution
- 将压缩建模为文本到图像生成，用扩散模型模拟多种编码器
- 在H.264/H.265/H.266下实现10%-40%码率节省
- 适合关注图像传输与存储效率的研究者和工程师
感知图像超分（SR）方法可恢复退化图像并生成清晰输出。实际中，这些输出通常需重新压缩以用于存储和传输。忽略再压缩会导致后续编码器引入额外伪影，影响效果。然而，联合优化超分与再压缩具有挑战性，因编码器不可导且配置多样。本文提出「通用再压缩感知感知超分」（VRPSR），使现有感知超分模型具备对多种压缩方式的适应能力。首先，将压缩建模为条件文本到图像生成任务，利用预训练扩散模型构建通用编码器模拟器；其次，设计专用于感知超分的训练策略，包括使用感知目标优化模拟器，并采用轻微压缩图像作为训练目标。实验表明，相较于Real-ESRGAN与S3Diff，VRPSR在H.264/H.265/H.266单帧（Intra）压缩下实现10%–40%的码率降低。此外，该方法还支持超分与再压缩后后处理模型的联合优化。
原文摘要 · Abstract (English)
Perceptual image super-resolution (SR) methods restore degraded images and produce sharp outputs. In practice, those outputs are usually recompressed for storage and transmission. Ignoring recompression is suboptimal as the downstream codec might add additional artifacts to restored images. However, jointly optimizing SR and recompression is challenging, as the codecs are not differentiable and vary in configuration. In this paper, we present \textbf{Versatile Recompression-Aware Perceptual Super-Resolution (VRPSR)}, which makes existing perceptual SR aware of versatile compression. First, we formulate compression as conditional text-to-image generation and utilize a pre-trained diffusion model to build a generalizable codec simulator. Next, we propose a set of training techniques tailored for perceptual SR, including optimizing the simulator using perceptual targets and adopting slightly compressed images as the training target. Empirically, our VRPSR achieves 10% - 40% bitrate savings based on Real-ESRGAN and S3Diff under H.264/H.265/H.266 single-picture (intra) compression. Besides, our VRPSR facilitates joint optimization of SR and the post-processing model after recompression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。