arXiv:2510.02307cs.CVcs.AI2025-10

让低分辨率图像生成更清晰,通过重校准噪声提升质量。

NoiseShift: Resolution-Aware Noise Recalibration for Better Low-Resolution Image Generation

  • 不改采样流程,只调整噪声索引以匹配低分辨率下的感知差异。
  • 在128x128下,SD3的FID从203降至171,显著提升图像质量。
  • 无需额外计算,插件式应用,适合资源受限场景使用。

文本到图像扩散模型在低于训练分辨率的条件下生成时性能下降。现有工作多关注高分辨率生成,而本文聚焦低分辨率生成,通过降低推理分辨率大幅减少计算开销。研究发现,噪声水平的网络条件在不同分辨率下导致感知退化不一致,引发训练-测试失配:相同噪声调度值在低分辨率下对应不同的感知破坏程度,干扰去噪器的时间步与噪声嵌入。为此,提出NoiseShift——一种无需训练的重校准方法,保持原始噪声采样不变,仅重新映射去噪器的噪声条件以恢复正向-反向一致性。通过小规模图像-文本对的粗到细校准,学习分辨率相关的噪声调度到条件噪声映射,缓解失配问题。应用于Stable Diffusion 3(SD3)、SD3.5和Flux-Dev后,低分辨率生成质量持续提升:SD3在128×128下FID由203降至171;SD3.5在LAION-COCO上从310降至277;即使已有时间偏移策略的Flux-Dev,在64×64下也从120降至113。关键优势在于无额外实现成本与推理开销。

原文摘要 · Abstract (English)

Text-to-image diffusion models often degrade when sampled at resolutions outside the final training resolution set. Prior work has largely emphasized higher resolution generation, enabling pretrained diffusion models to extrapolate beyond the resolutions seen during training. In this work, we instead target lower-resolution generation, performing inference at reduced resolution to significantly cut computational cost. We show that network conditioning of the noise level induces a train-test mismatch that directly degrades low-resolution generation: the same scheduled noise level can correspond to a different perceptual corruption level at lower resolutions, mis-calibrating the denoiser timestep and noise embedding. To this end, we propose NoiseShift, a training-free recalibration method that keeps the original noise sampling schedule unchanged and instead re-indexes the noise conditioning of the denoiser to restore local forward-reverse consistency. Using a lightweight coarse-to-fine calibration on a small set of image-text pairs, NoiseShift learns a resolution-specific mapping from scheduler noise to conditioning noise, reducing train-test mismatch and improving lower-resolution generation quality. When NoiseShift is applied to Stable Diffusion 3 (SD3), Stable Diffusion 3.5 (SD3.5), and Flux-Dev, generation quality at low resolutions improves consistently. Particularly, SD3 generation at 128x128 resolution gets an improved FID score from 203 to 171, and SD3.5 gets an improved FID score from 310 to 277 on LAION-COCO. Even Flux-Dev which already implements a complementary time-shifting strategy gets a modest boost from NoiseShift with an improved FID score from 120 to 113 at 64x64 resolution. More importantly, NoiseShift achieves such improvements with minimal implementation changes and no additional inference overhead.

图像生成扩散模型低分辨率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。