让单步图像超分更准更可控，通过时间感知机制激活扩散模型的生成潜力。
Time-Aware One Step Diffusion Network for Real-World Image Super-Resolution
- 引入时间感知VAE编码器，根据不同时间步生成差异化的隐空间特征。
- 设计时间感知VSD损失，使学生模型在不同时间步上对齐教师模型的生成先验。
- 仅用一步即可实现高保真与高真实感之间的可调控平衡，适合实际应用。
基于扩散模型的现实世界图像超分辨率（Real-ISR）方法表现优异。为实现高效超分，许多工作采用变分分数蒸馏（VSD）将预训练的Stable Diffusion（SD）模型蒸馏为单步超分模型，并固定时间步。然而，由于SD在不同时间步具有不同的生成先验，固定时间步难以充分挖掘其生成能力，导致性能受限。为此，本文提出时间感知的一步扩散网络（TADSR）。首先引入时间感知VAE编码器，根据时间步将同一图像映射到不同隐空间特征；通过动态调整时间步与隐特征的联合分布，使学生模型更好地匹配预训练SD的输入模式分布，从而更有效利用其生成能力。进一步提出时间感知VSD损失，弥合学生模型与教师模型的时间步差异，提供更一致的时间条件生成先验引导。此外，该方法能自然实现保真度与真实感之间的可控权衡。实验表明，本方法仅用一步即可达到最先进性能并支持可控输出。代码已开源。
原文摘要 · Abstract (English)
Diffusion-based real-world image super-resolution (Real-ISR) methods have demonstrated impressive performance.To achieve efficient Real-ISR, many works employ Variational Score Distillation (VSD) to distill pre-trained stable-diffusion (SD) model for one-step SR with a fixed timestep. However, since SD will perform different generative priors at different timesteps, a fixed timestep is difficult for these methods to fully leverage the generative priors in SD, leading to suboptimal performance.To address this, we propose a \textbf{T}ime-\textbf{A}ware one-step \textbf{D}iffusion Network for Real-ISR (\textbf{TADSR}). We first introduce a Time-Aware VAE Encoder, which projects the same image into different latent features based on timesteps.Through joint dynamic variation of timesteps and latent features, the student model can better align with the input pattern distribution of the pre-trained SD, thereby enabling more effective utilization of SD's generative capabilities.To better activate the generative prior of SD at different timesteps, we propose a Time-Aware VSD loss that bridges the timesteps of the student model and those of the teacher model, thereby producing more consistent generative prior guidance conditioned on timesteps. Additionally, though utilizing the generative prior in SD at different timesteps, our method can naturally achieve \textbf{controllable trade-offs between fidelity and realism} by changing the timestep.Experimental results demonstrate that our method achieves both state-of-the-art performance and controllable SR results with only a single step. The source codes are released at https://github.com/zty557/TADSR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。