arXiv:2411.16738cs.CVcs.AI2024-11CVPR被引 20

扩散模型生成时易复现训练数据,本文提出新方法避免记忆。

Classifier-Free Guidance inside the Attraction Basin May Cause Memorization

  • 通过延迟应用无分类器引导,避开记忆吸引盆地。
  • 在多个场景中验证可有效降低记忆现象,保持图像质量。
  • 适合关注版权与隐私的生成模型研究者使用。

扩散模型容易精确复现训练数据中的图像,这可能导致版权侵权或隐私泄露。本文从新视角分析记忆现象,提出一种简单有效的方法:在去噪过程中延迟无分类器引导的应用,直到理想过渡点再启用,从而引导扩散轨迹远离记忆图像所在吸引盆地。进一步提出反向引导技术,更早脱离吸引盆地。实验表明,在多种记忆发生场景中,该方法能有效抑制记忆,同时生成高质量且符合条件的图像。

原文摘要 · Abstract (English)

Diffusion models are prone to exactly reproduce images from the training data. This exact reproduction of the training data is concerning as it can lead to copyright infringement and/or leakage of privacy-sensitive information. In this paper, we present a novel perspective on the memorization phenomenon and propose a simple yet effective approach to mitigate it. We argue that memorization occurs because of an attraction basin in the denoising process which steers the diffusion trajectory towards a memorized image. However, this can be mitigated by guiding the diffusion trajectory away from the attraction basin by not applying classifier-free guidance until an ideal transition point occurs from which classifier-free guidance is applied. This leads to the generation of non-memorized images that are high in image quality and well-aligned with the conditioning mechanism. To further improve on this, we present a new guidance technique, opposite guidance, that escapes the attraction basin sooner in the denoising process. We demonstrate the existence of attraction basins in various scenarios in which memorization occurs, and we show that our proposed approach successfully mitigates memorization.

扩散模型记忆问题无分类器引导隐私安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。