用信息论正则化提升神经图像压缩的效率与泛化能力
An Information-Theoretic Regularizer for Lossy Neural Image Compression
- 通过最大化条件源熵来优化隐变量压缩
- 在多种结构和未见域上均能进一步降低比特率
- 方法可解释、即插即用且推理无开销
有损图像压缩网络旨在最小化图像的隐变量熵，同时满足特定失真约束。然而，由于学习量化隐表示的特性，优化神经网络存在挑战。本文的核心发现是：最小化隐变量熵在一定程度上等价于最大化条件源熵，这一观点基于信息论等式。基于此，我们提出一种新型结构正则化方法，将负条件源熵引入训练目标，从而提升优化效率和模型泛化能力。所提信息论正则化器具有可解释性、即插即用性，且不增加推理开销。大量实验表明，该方法在多种压缩结构及未见领域中均能有效正则化模型，并进一步压缩隐变量表示的比特数。
原文摘要 · Abstract (English)
Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints. However, optimizing the neural network can be challenging due to its nature of learning quantized latent representations. In this paper, our key finding is that minimizing the latent entropy is, to some extent, equivalent to maximizing the conditional source entropy, an insight that is deeply rooted in information-theoretic equalities. Building on this insight, we propose a novel structural regularization method for the neural image compression task by incorporating the negative conditional source entropy into the training objective, such that both the optimization efficacy and the model's generalization ability can be promoted. The proposed information-theoretic regularizer is interpretable, plug-and-play, and imposes no inference overheads. Extensive experiments demonstrate its superiority in regularizing the models and further squeezing bits from the latent representation across various compression structures and unseen domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。