通过约束噪声空间几何结构,提升快速生成图像的多样性。
Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling

- 在低维噪声流形上进行黎曼优化,保持生成质量的同时增强多样性。
- 仅需10次迭代即可收敛,比现有方法减少90%以上计算量。
- 无需额外质量控制损失,适合对生成多样性要求高的场景。
少步微调扩散模型可快速生成高质量图像,但常因提示词多样性不足,导致不同随机种子生成结果高度相似。推理时优化初始噪声是恢复多样性的有效方法,但现有方法在无约束的欧氏空间中直接更新噪声,忽略了高斯先验的几何特性及模型对噪声频率的敏感性。因此需引入辅助质量控制目标,增加计算开销与超参数调优,且更新幅度受限以避免退化。本文提出训练无关的MoNO方法,在一个低维、质量稳定的噪声流形上进行流形约束噪声优化。该方法依次优化每个新初始噪声,使其预测视觉特征能补充先前生成结果;通过仿射低频球面上的黎曼更新,天然保留先验似然并修复不稳定的高频成分。这使得可采用大测地线步长,无需辅助质量控制目标,且迭代次数显著减少。多个微调文本到图像扩散模型实验表明,MoNO在保持图像质量的同时,持续提升每提示词的生成多样性。
原文摘要 · Abstract (English)
Few-step distilled diffusion models generate high-quality images quickly, but often lose per-prompt diversity, producing near-identical samples across random seeds. Optimizing the initial noise at inference time offers an appealing way to recover this diversity, yet existing methods directly update the initial noise in an unconstrained Euclidean space, ignoring both the geometry of the Gaussian prior and the model's sensitivity to noise frequencies. They therefore introduce auxiliary quality-control objectives to maintain generation fidelity, adding compute and weighting hyperparameters while still requiring conservative updates to prevent degradation. In this work, we propose MoNO, a training-free method that performs Manifold-constrained Noise Optimization on a low-dimensional, quality-stabilizing noise manifold. MoNO sequentially optimizes each new initial noise so that its predicted visual feature complements previous generations, while Riemannian updates on an affine low-frequency sphere preserve prior likelihood and fix unstable high-frequency components by construction. This enables large geodesic steps, removes the need for auxiliary quality-control objectives, and converges in far fewer iterations than prior noise-optimization methods. Experiments with multiple distilled text-to-image diffusion models show that MoNO consistently improves per-prompt diversity while maintaining image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。