提出频谱对齐方法，缓解扩散模型采样时的误差累积问题。
Spectral Prior for Reducing Exposure Bias in Diffusion Models
- 通过频谱校准中间预测，修正训练与推理间的频率偏差。
- 在多种扩散模型上提升生成质量，平均提升0.5~1.2个FID分数。
- 轻量级设计，兼容现有指导机制，适合实际部署场景。
扩散模型在迭代采样过程中常因误差累积而表现下降，这被称为暴露偏差。我们揭示了训练与推理间存在系统性的频率依赖性差异，可解释为频率依赖的信噪比（SNR）误差。关键发现是，这种失配方向随模型和时间步变化，表明固定修正规则无法通用。为此，我们提出频谱对齐（Spectral Alignment, SPA），一种基于引导的轻量级方法，通过将中间预测的功率谱校准到预先计算的先验谱来实现优化。该方法分两阶段：(1) 离线从训练数据拟合参数化谱模型；(2) 推理时通过高效的FFT-based梯度计算进行引导。SPA引入仅3-4%的额外计算开销，且与无分类器引导（CFG）互补。我们在多种架构上验证其有效性，包括像素空间模型（DDPM、ADM）、潜在扩散模型（SD2.0、SDXL）以及流匹配模型（SD3.5、FLUX）。代码已开源于https://github.com/SonyResearch/SPA。
原文摘要 · Abstract (English)
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies between training and inference, which can be interpreted as frequency-dependent SNR error. Crucially, the direction of this mismatch varies across models and timesteps, indicating that fixed correction rules do not generalize. We propose Spectral Alignment (SPA), a lightweight, guidance-based method that calibrates the power spectrum of intermediate predictions to a pre-computed prior. Our approach consists of two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference-time guidance via efficient FFT-based gradient computation. SPA introduces minimal computational overhead (3-4\%) and is complementary to Classifier-Free Guidance (CFG). We demonstrate consistent improvements across diverse architectures, from pixel-space models (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX). Our implementation is available at https://github.com/SonyResearch/SPA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。