arXiv:2605.23902cs.CV2026-05被引 2

用像素扩散统一解码与超分,生成速度提升6倍

PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

论文配图:PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion
图 1 · 摘自论文原文
  • 将潜空间解码转为条件像素扩散,直接生成高分辨率图像
  • 512×512潜变量1秒内生成2048×2048像素图,峰值显存13GB
  • 支持多种潜变量,适配主流文生图系统,适合追求速度的开发者

现有高分辨率文生图系统多在紧凑潜空间中生成,并通过解码器映射回像素。但传统解码器以重建为导向,优化目标是逆编码而非细节合成,在兆像素尺度下效率低下。为此,本文提出PiD:一种像素扩散解码器,将潜空间解码重构为条件像素扩散,统一解码与上采样。通过在高分辨率像素空间直接去噪,PiD可实现4倍至8倍超分辨率生成且延迟极低。针对潜变量条件,设计轻量级时序感知适配器,将噪声潜变量注入像素扩散主干,支持部分去噪潜变量输入并提前终止潜空间扩散过程。进一步采用DMD2进行模型蒸馏,推理仅需4步。PiD兼容传统VAE潜变量及近期RAE模型中的语义潜变量(如SigLIP、DINOv2)。在消费级RTX 5090上,512×512潜变量解码至2048×2048像素耗时不足1秒,峰值显存13GB;在GB200 GPU上最快达210毫秒,较级联扩散超分流水线快约6倍,且视觉质量更优。

原文摘要 · Abstract (English)

Most practical high-resolution text-to-image systems, including latent diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the generated latents back to pixels. Yet the latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details, and becomes increasingly costly at megapixel scale. This drawback calls for a more expressive and efficient decoding paradigm. Motivated by recent progress in scalable pixel-space diffusion, we introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion, unifying decoding and upsampling into one generative module. By denoising directly in high-resolution pixel space, PiD synthesizes $4\times$ and even $8\times$ upscaled images with low latency. For latent conditioning, a lightweight sigma-aware adapter injects noise-corrupted latents into the pixel diffusion backbone, enabling PiD to decode partially denoised latents and terminate the latent diffusion process early. To further improve efficiency, we distill the model using DMD2, reducing inference to just 4 steps. PiD applies to both conventional VAE latents and semantic latents (e.g., SigLIP, DINOv2) used in recent RAE-based models. PiD decodes latents of $512 \times 512$ images into $2048 \times 2048$ pixels in under 1 second with 13 GB peak memory on a consumer RTX 5090, and as fast as 210 ms on a GB200 GPU, about $6\times$ faster than cascaded diffusion-based super-resolution pipelines with better visual fidelity.

扩散模型图像生成超分辨率高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。