arXiv:2511.10629cs.CV2025-11被引 1

让扩散模型快速生成高清图,不改模型还能省时3倍。

One Small Step in Latent, One Giant Leap for Pixels: Fast Latent Upscale Adapter for Your Diffusion Models

  • 在隐空间直接升频,一步完成,无需额外训练
  • 2倍和4倍升频仅增0.42秒延迟,比像素空间快3倍
  • 通用性强,适配多种VAE,部署无需重训

扩散模型难以突破训练分辨率,直接高分辨率采样效率低,而后期超分会引入伪影并增加延迟。本文提出潜空间升频适配器(LUA),轻量模块在生成器的潜在表示上直接执行超分,无需修改基础模型或增加扩散阶段。通过一次前向传播即可实现高分辨率合成,采用共享的Swin式主干与尺度特定的像素洗牌头,支持2x和4x升频。该方法在保持与原生高分辨率生成相近感知质量的同时,将解码与升频时间降低近3倍(从512到1024像素生成仅增加0.42秒,相较同架构的像素空间超分减少至1.87秒)。此外,LUA在不同VAE的潜空间中表现良好,可跨模型部署且无需重新训练。大量实验表明,其在真实性和效率间取得平衡,为现代扩散模型提供高效、可扩展的高清图像生成路径。

原文摘要 · Abstract (English)

Diffusion models struggle to scale beyond their training resolutions, as direct high-resolution sampling is slow and costly, while post-hoc image super-resolution (ISR) introduces artifacts and additional latency by operating after decoding. We present the Latent Upscaler Adapter (LUA), a lightweight module that performs super-resolution directly on the generator's latent code before the final VAE decoding step. LUA integrates as a drop-in component, requiring no modifications to the base model or additional diffusion stages, and enables high-resolution synthesis through a single feed-forward pass in latent space. A shared Swin-style backbone with scale-specific pixel-shuffle heads supports 2x and 4x factors and remains compatible with image-space SR baselines, achieving comparable perceptual quality with nearly 3x lower decoding and upscaling time (adding only +0.42 s for 1024 px generation from 512 px, compared to 1.87 s for pixel-space SR using the same SwinIR architecture). Furthermore, LUA shows strong generalization across the latent spaces of different VAEs, making it easy to deploy without retraining from scratch for each new decoder. Extensive experiments demonstrate that LUA closely matches the fidelity of native high-resolution generation while offering a practical and efficient path to scalable, high-fidelity image synthesis in modern diffusion pipelines.

扩散模型超分潜空间高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。