提出动态时空采样策略,让扩散模型超分更快更准
Accelerating Diffusion-based Super-Resolution with Dynamic Time-Spatial Sampling
- 根据高频细节恢复需求,动态分配不同阶段的迭代次数
- 仅用一半步数就达到当前最优性能,MUSIQ提升0.2至3.0
- 无需额外训练,适合追求高效高质图像超分的开发者
扩散模型在建模复杂分布方面表现优异,在超分辨率任务中实现了出色的感知质量。然而,现有基于扩散的超分方法通常计算成本高,需大量迭代步骤完成训练与推理。现有加速技术如知识蒸馏和求解器优化普遍缺乏任务针对性,未能充分利用低层任务(如超分辨率)的特性。本研究分析了扩散超分方法在频域与空域的特性,揭示了高频信号恢复在时间与空间上的依赖规律:高频细节在扩散过程的早期与晚期集中优化效果最佳,而纹理区域则需自适应去噪策略。基于此,我们提出无需额外训练的时空感知采样策略(TSS),融合时间动态采样(TDS)与空间动态采样(SDS),分别针对纹理优化与内容自适应调整。在多个基准测试中,TSS显著减少迭代次数,实现当前最优性能,相较现有加速方法以一半步数提升MUSIQ得分0.2–3.0。
原文摘要 · Abstract (English)
Diffusion models have gained attention for their success in modeling complex distributions, achieving impressive perceptual quality in SR tasks. However, existing diffusion-based SR methods often suffer from high computational costs, requiring numerous iterative steps for training and inference. Existing acceleration techniques, such as distillation and solver optimization, are generally task-agnostic and do not fully leverage the specific characteristics of low-level tasks like super-resolution (SR). In this study, we analyze the frequency- and spatial-domain properties of diffusion-based SR methods, revealing key insights into the temporal and spatial dependencies of high-frequency signal recovery. Specifically, high-frequency details benefit from concentrated optimization during early and late diffusion iterations, while spatially textured regions demand adaptive denoising strategies. Building on these observations, we propose the Time-Spatial-aware Sampling strategy (TSS) for the acceleration of Diffusion SR without any extra training cost. TSS combines Time Dynamic Sampling (TDS), which allocates more iterations to refining textures, and Spatial Dynamic Sampling (SDS), which dynamically adjusts strategies based on image content. Extensive evaluations across multiple benchmarks demonstrate that TSS achieves state-of-the-art (SOTA) performance with significantly fewer iterations, improving MUSIQ scores by 0.2 - 3.0 and outperforming the current acceleration methods with only half the number of steps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。