通过优化初始噪声提升文本生成图像的多样性,效果显著且稳定。
Diverse Text-to-Image Generation via Contrastive Noise Optimization
- 在噪声空间设计对比损失,预处理初始噪声以促进多样性
- 在多个主流模型上实现更优的质量-多样性权衡,优于现有方法
- 无需精细调参,适合追求多样输出的图像生成应用
文本到图像扩散模型在生成高保真图像方面表现优异,主要依赖文本引导推理。然而,强文本引导常导致输出模式单一、多样性不足。现有方法多在推理中优化中间潜在表示或文本条件,但增益有限且对超参数敏感。本文提出对比噪声优化(Contrastive Noise Optimization),从新视角解决多样性问题:不调整中间潜变量,而是优化初始噪声。具体地,在Tweedie数据空间定义对比损失,对一批噪声潜在变量进行优化。该方法在批量内排斥相似样本以最大化多样性,同时锚定参考样本以保持生成质量。我们进一步提供理论分析支持其有效性。在多个文本到图像骨干模型上的实验表明,本方法在质量-多样性帕累托前沿上表现更优,且对超参数不敏感。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models have demonstrated impressive performance in generating high-fidelity images, largely enabled by text-guided inference. However, this advantage often comes with a critical drawback: limited diversity, as outputs tend to collapse into similar modes under strong text guidance. Existing approaches typically optimize intermediate latents or text conditions during inference, but these methods deliver only modest gains or remain sensitive to hyperparameter tuning. In this work, we introduce Contrastive Noise Optimization, a simple yet effective method that addresses the diversity issue from a distinct perspective. Unlike prior techniques that adapt intermediate latents, our approach shapes the initial noise to promote diverse outputs. Specifically, we develop a contrastive loss defined in the Tweedie data space and optimize a batch of noise latents. Our contrastive optimization repels instances within the batch to maximize diversity while keeping them anchored to a reference sample to preserve fidelity. We further provide theoretical insights into the mechanism of this preprocessing to substantiate its effectiveness. Extensive experiments across multiple T2I backbones demonstrate that our approach achieves a superior quality-diversity Pareto frontier while remaining robust to hyperparameter choices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。