arXiv:2606.01608cs.CV2026-06

用语义与像素双表示,实现超低比特率下保真度提升

Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression

论文配图:Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression
图 1 · 摘自论文原文
  • 设计三编码器结构,融合语义与像素特征补偿压缩损失
  • 在低于0.03 bpp时,重建图像保持视觉真实与像素级一致
  • 适合需要极低码率且高保真的图像压缩场景

现有极端压缩方法往往在感知保真与视觉真实上过度追求,而忽视像素级准确性,导致重构图像与原图差异明显。超低比特率图像压缩不仅要求极小的表示体积,还需确保重构图像在语义和像素层面均忠实于源图像。为此,我们提出SPRDiff,一种基于扩散模型的压缩方法,充分结合语义与像素表示,显著提升超低比特率下的重建保真度。具体地,设计三编码器架构,利用预训练的失真导向与语义导向编码器提取的高保真特征,弥补冻结VAE编码器的表达局限,优化潜在表示压缩与熵建模。进一步引入失真感知重建模块,实现双特征提取:生成保留主体结构的粗略重构,并提供精准的语义与像素级条件信号以引导扩散模型。大量实验表明,在基准数据集上,本方法在低于0.03 bpp的极低比特率下,优于当前最优方案,在率失真感知权衡上表现更优,有效同时保持感知质量与像素级一致性。代码与模型将开源于https://github.com/cshw2021/SPRDiff。

原文摘要 · Abstract (English)

Most existing extreme compression methods fail to achieve an optimal rate-distortion-perception trade-off, as they typically prioritize perceptual fidelity and visual realism over pixel-level accuracy. Consequently, the resulting reconstructions often deviate noticeably from the originals. Ultra-low bitrate image compression is therefore crucial-not only for producing extremely compact representations but also for ensuring that reconstructed images remain semantically coherent and faithful to the source at the pixel level. To this end, we propose SPRDiff, a diffusion-based compression method that fully leverages both semantic and pixel representations, thereby enhancing reconstruction fidelity under ultra-low bitrate constraints. Specifically, we develop a triple-encoder architecture that utilizes high-fidelity features from the pretrained distortion-oriented and semantic-oriented encoders to compensate for the limited representations extracted by the frozen VAE encoder, thereby improving latent compression and entropy modeling. To further enhance the reconstruction fidelity of diffusion models, we introduce a distortion-aware reconstruction module with dual feature extraction. This module not only generates a coarse reconstruction that preserves the main structures, but also provides practical and accurate semantic- and pixel-level conditional signals to guide the diffusion model. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches in the rate-distortion-perception tradeoff at extremely low bitrates (below 0.03 bpp), effectively preserving both perceptual quality and pixel-wise fidelity in the reconstructed images. We will release the source code and trained models at https://github.com/cshw2021/SPRDiff.

图像压缩扩散模型超低比特率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。