arXiv:2409.11111eess.IVcs.CV2024-09AAAI被引 4

用少量样本让压缩模型适应新图像域,性能接近顶级编码标准。

Few-Shot Domain Adaptation for Learned Image Compression

论文配图:Few-Shot Domain Adaptation for Learned Image Compression
图 1 · 摘自论文原文
  • 在预训练压缩模型中加入轻量适配器,按通道重分配特征。
  • 仅用25个目标域样本,就达到H.266/VVC级压缩效果。
  • 传输参数不足2%,适合资源受限场景的快速部署。

学习型图像压缩(LIC)已实现顶尖的率失真性能,被视为下一代图像压缩技术的希望。然而,预训练的LIC模型在处理非训练域图像时性能显著下降,表明其泛化能力不足。为此,我们提出一种面向LIC的少样本域自适应方法,通过将即插即用适配器集成到预训练模型中来解决该问题。受潜在通道与频域成分类比的启发,我们分析了LIC中的域差距,发现非训练域图像会破坏预训练的通道分解结构。因此,我们引入基于卷积和低秩的通道重分配方法,该方法轻量且兼容主流LIC方案。在多个域和多种代表性LIC架构上的大量实验表明,该方法显著提升了预训练模型性能,在仅使用25个目标域样本的情况下,达到了与H.266/VVC内编码相当的水平。此外,其性能可媲美全模型微调,但仅需传输不到2%的参数。

原文摘要 · Abstract (English)

Learned image compression (LIC) has achieved state-of-the-art rate-distortion performance, deemed promising for next-generation image compression techniques. However, pre-trained LIC models usually suffer from significant performance degradation when applied to out-of-training-domain images, implying their poor generalization capabilities. To tackle this problem, we propose a few-shot domain adaptation method for LIC by integrating plug-and-play adapters into pre-trained models. Drawing inspiration from the analogy between latent channels and frequency components, we examine domain gaps in LIC and observe that out-of-training-domain images disrupt pre-trained channel-wise decomposition. Consequently, we introduce a method for channel-wise re-allocation using convolution-based adapters and low-rank adapters, which are lightweight and compatible to mainstream LIC schemes. Extensive experiments across multiple domains and multiple representative LIC schemes demonstrate that our method significantly enhances pre-trained models, achieving comparable performance to H.266/VVC intra coding with merely 25 target-domain samples. Additionally, our method matches the performance of full-model finetune while transmitting fewer than $2\%$ of the parameters.

图像压缩少样本学习域自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。