用可学习的参考特征动态编码图像,显著提升压缩效率。
Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression
- 从外部字典中合成条件隐变量,指导图像隐空间编码。
- 在基准数据集上最高提升1.2 dB,仅增加0.5%比特开销。
- 适合追求高精度压缩且关注模型稳定性与泛化能力的研究者。
本文研究如何从外部字典中动态合成条件参考,以实现输入图像在隐空间中的条件编码,并端到端学习条件隐变量合成与编码模块。方法通过多阶段构建通用图像特征字典,包含改进的空间金字塔池化、降维和多尺度特征聚类。针对每张输入图像,学习从字典中选择并合成相关特征,生成条件隐变量,显著增强模型捕捉和利用图像源相关性的能力。该过程采用基于相关性的特征匹配与对齐策略,包括条件隐变量匹配(CLM)模块和条件隐变量合成(CLS)模块。合成的隐变量用于引导编码过程,通过利用输入图像与参考字典间的相关性实现更高效的压缩。理论分析表明,所提出的条件隐变量编码(CLC)方法对字典样本扰动和所选条件隐变量具有鲁棒性,误差界随字典规模呈对数级增长,确保大规模多样化字典下的稳定性。在基准数据集上的实验结果表明,新方法在编码性能上大幅提升(最高达1.2 dB),仅带来约0.5%比特每像素的额外开销。代码已公开于 https://github.com/ydchen0806/CLC。
原文摘要 · Abstract (English)
In this paper, we study how to synthesize a dynamic reference from an external dictionary to perform conditional coding of the input image in the latent domain and how to learn the conditional latent synthesis and coding modules in an end-to-end manner. Our approach begins by constructing a universal image feature dictionary using a multi-stage approach involving modified spatial pyramid pooling, dimension reduction, and multi-scale feature clustering. For each input image, we learn to synthesize a conditioning latent by selecting and synthesizing relevant features from the dictionary, which significantly enhances the model's capability in capturing and exploring image source correlation. This conditional latent synthesis involves a correlation-based feature matching and alignment strategy, comprising a Conditional Latent Matching (CLM) module and a Conditional Latent Synthesis (CLS) module. The synthesized latent is then used to guide the encoding process, allowing for more efficient compression by exploiting the correlation between the input image and the reference dictionary. According to our theoretical analysis, the proposed conditional latent coding (CLC) method is robust to perturbations in the external dictionary samples and the selected conditioning latent, with an error bound that scales logarithmically with the dictionary size, ensuring stability even with large and diverse dictionaries. Experimental results on benchmark datasets show that our new method improves the coding performance by a large margin (up to 1.2 dB) with a very small overhead of approximately 0.5\% bits per pixel. Our code is publicly available at https://github.com/ydchen0806/CLC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。