通过调整初始噪声，让扩散模型更早逃离记忆陷阱，提升生成安全性和提示一致性。
Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models
- 改变生成初始噪声的分布，促使模型更早脱离记忆区域。
- 实验显示新方法在保持提示对齐的前提下，显著降低图像记忆现象。
- 适合关注生成模型隐私与版权问题的研究者和开发者。
尽管文本到图像的扩散模型具有出色的生成能力，但它们常会记忆并复现训练数据，引发隐私与版权担忧。近期研究指出，这种记忆源于一个吸引盆——即应用无分类器引导（CFG）时，去噪轨迹会被拉向记忆输出的区域。为此，有方法提出延迟应用CFG以使轨迹先逃出该区域，但常导致生成图像与输入提示对齐不佳。本文发现初始噪声样本对逃逸时间有关键影响：不同初始噪声导致不同的逃逸时机。基于此，我们提出两种策略，通过集体或个体调整初始噪声，寻找能促进早期逃逸的样本。该方法显著减少记忆现象，同时保持图像与文本的对齐性。
原文摘要 · Abstract (English)
Despite their impressive generative capabilities, text-to-image diffusion models often memorize and replicate training data, prompting serious concerns over privacy and copyright. Recent work has attributed this memorization to an attraction basin-a region where applying classifier-free guidance (CFG) steers the denoising trajectory toward memorized outputs-and has proposed deferring CFG application until the denoising trajectory escapes this basin. However, such delays often result in non-memorized images that are poorly aligned with the input prompts, highlighting the need to promote earlier escape so that CFG can be applied sooner in the denoising process. In this work, we show that the initial noise sample plays a crucial role in determining when this escape occurs. We empirically observe that different initial samples lead to varying escape times. Building on this insight, we propose two mitigation strategies that adjust the initial noise-either collectively or individually-to find and utilize initial samples that encourage earlier basin escape. These approaches significantly reduce memorization while preserving image-text alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。