arXiv:2603.13162eess.IVcs.CV2026-03被引 1

用Transformer实现32倍压缩,速度比现有方法快30倍

DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression

  • 用扩散Transformer替代U-Net,直接在32倍下采样空间中重建图像
  • 实现顶尖视觉质量,2048×2048图像可在16GB显卡上解码
  • 三重对齐机制支持单步推理,无需文本提示,适合实时应用

基于扩散模型的图像压缩虽具出色感知保真度,但受限于高昂采样开销与高内存占用。现有方法多采用U-Net架构,层级下采样使扩散过程仅在浅层潜在空间(通常仅8倍空间下采样)进行,导致计算冗余。相比之下,传统VAE编码器可达到16至64倍下采样,引发关键问题:扩散能否在如此紧凑的潜在空间中有效运行而不损失重建质量?为此,本文提出DiT-IC,一种面向图像压缩的对齐扩散Transformer,将原生多步文本生成扩散Transformer重构为单步重建模型。通过三项核心对齐机制:(1) 方差引导重建流,根据潜在不确定性自适应调整去噪强度;(2) 自蒸馏对齐,保持与编码器定义的潜在几何一致性以支持单步扩散;(3) 潜在条件引导,以语义对齐的潜在向量替代文本提示,实现无文本推理。该设计使DiT-IC在保持最先进感知质量的同时,解码速度较现有扩散编码器提升高达30倍,内存消耗显著降低。尤为突出的是,其可在16GB笔记本显卡上完成2048×2048图像的重建。

原文摘要 · Abstract (English)

Diffusion-based image compression has recently shown outstanding perceptual fidelity, yet its practicality is hindered by prohibitive sampling overhead and high memory usage. Most existing diffusion codecs employ U-Net architectures, where hierarchical downsampling forces diffusion to operate in shallow latent spaces (typically with only 8x spatial downscaling), resulting in excessive computation. In contrast, conventional VAE-based codecs work in much deeper latent domains (16x - 64x downscaled), motivating a key question: Can diffusion operate effectively in such compact latent spaces without compromising reconstruction quality? To address this, we introduce DiT-IC, an Aligned Diffusion Transformer for Image Compression, which replaces the U-Net with a Diffusion Transformer capable of performing diffusion in latent space entirely at 32x downscaled resolution. DiT-IC adapts a pretrained text-to-image multi-step DiT into a single-step reconstruction model through three key alignment mechanisms: (1) a variance-guided reconstruction flow that adapts denoising strength to latent uncertainty for efficient reconstruction; (2) a self-distillation alignment that enforces consistency with encoder-defined latent geometry to enable one-step diffusion; and (3) a latent-conditioned guidance that replaces text prompts with semantically aligned latent conditions, enabling text-free inference. With these designs, DiT-IC achieves state-of-the-art perceptual quality while offering up to 30x faster decoding and drastically lower memory usage than existing diffusion-based codecs. Remarkably, it can reconstruct 2048x2048 images on a 16 GB laptop GPU.

图像压缩扩散模型Transformer高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。