用代码本替换扩散模型噪声，生成图像同时输出可压缩比特流。
Compressed Image Generation with Denoising Diffusion Codebook Models
- 用预定义代码本中的高斯向量替代标准噪声采样。
- 小代码本下仍保持高质量图像生成与多样性。
- 适用于图像压缩与修复等条件生成任务，效果领先。
我们提出一种基于去噪扩散模型（DDMs）的新生成方法，可在生成高质量图像的同时，输出其无损压缩的比特流表示。该方法通过用预定义代码本中的固定独立同分布高斯向量替代反向扩散过程中的标准高斯噪声采样实现。令人惊讶的是，即使使用极小的代码本，所提出的去噪扩散代码本模型（DDCM）仍能保持与标准DDM相当的样本质量与多样性。我们利用DDCM，选择最匹配目标图像的代码本噪声，将生成模型转化为高效的有损图像编解码器，在感知图像压缩方面达到当前最优水平。更一般地，通过设定其他噪声选择规则，我们的压缩方法可扩展至任意条件图像生成任务（如图像修复），生成图像的同时附带紧凑的比特流表示。本文还提供了对所提压缩条件生成方案的数学解释，建立了其与所考虑任务后验采样器的分数基近似之间的联系。
原文摘要 · Abstract (English)
We present a novel generative approach based on Denoising Diffusion Models (DDMs), which produces high-quality image samples along with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaussian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed Denoising Diffusion Codebook Model (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match a given image, converting our generative model into a highly effective lossy image codec achieving state-of-the-art perceptual image compression results. More generally, by setting other noise selections rules, we extend our compression method to any conditional image generation task (e.g., image restoration), where the generated images are produced jointly with their condensed bit-stream representations. Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing a connection with score-based approximations of posterior samplers for the tasks considered.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。