提出首个面向图像压缩的扩散模型CoD，可实现超低码率下的高效压缩与生成。
CoD: A Diffusion Foundation Model for Image Compression
- 从头训练压缩导向的扩散模型，支持端到端优化压缩与生成
- 在0.0039 bpp下达到当前最优性能，比Stable Diffusion快300倍训练
- 适用于多种基于扩散的编解码器，适合研究低码率压缩的学者
现有扩散编解码器多基于如Stable Diffusion等文本到图像的扩散基础模型，但文本条件对压缩不友好，尤其在超低码率下限制了性能。为此，我们提出首个压缩导向的扩散基础模型CoD，从头训练以实现压缩与生成的端到端优化。CoD并非固定编解码器，而是一个通用基础模型，可支持多种扩散编解码器。其优势包括：高压缩效率——在下游编解码器DiffC中替换Stable Diffusion后，在0.0039 bpp下达到当前最优性能；低成本可复现训练——仅需约20个A100 GPU天（相比Stable Diffusion的6,250天），且使用完全开源的仅图像数据集；提供新洞见——发现像素空间扩散可实现接近VTM的PSNR并保持高感知质量，且用更少参数超越基于GAN的编解码器。代码已公开于https://github.com/microsoft/GenCodec/tree/main/CoD。
原文摘要 · Abstract (English)
Existing diffusion codecs typically build on text-to-image diffusion foundation models like Stable Diffusion. However, text conditioning is suboptimal from a compression perspective, hindering the potential of downstream diffusion codecs, particularly at ultra-low bitrates. To address it, we introduce \textbf{CoD}, the first \textbf{Co}mpression-oriented \textbf{D}iffusion foundation model, trained from scratch to enable end-to-end optimization of both compression and generation. CoD is not a fixed codec but a general foundation model designed for various diffusion-based codecs. It offers several advantages: \textbf{High compression efficiency}, replacing Stable Diffusion with CoD in downstream codecs like DiffC achieves SOTA results, especially at ultra-low bitrates (e.g., 0.0039 bpp); \textbf{Low-cost and reproducible training}, 300$\times$ faster training than Stable Diffusion ($\sim$ 20 vs. $\sim$ 6,250 A100 GPU days) on entirely open image-only datasets; \textbf{Providing new insights}, e.g., We find pixel-space diffusion can achieve VTM-level PSNR with high perceptual quality and can outperform GAN-based codecs using fewer parameters. We hope CoD lays the foundation for future diffusion codec research. Codes are released at https://github.com/microsoft/GenCodec/tree/main/CoD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。