通过注入语义特征抑制扩散模型记忆训练数据，提升隐私安全。
CAPTAIN: Semantic Feature Injection for Memorization Mitigation in Text-to-Image Diffusion Models
- 在去噪过程中直接修改潜在特征，避免依赖引导参数调节。
- 相比基线方法，记忆现象显著降低，同时保持提示词对齐度。
- 适合关注生成模型隐私与版权问题的研究者和开发者。
扩散模型可能无意中复现训练数据，随着大规模部署引发隐私与版权担忧。现有推理阶段缓解方法多依赖无分类器引导（CFG）或扰动提示嵌入，但常难以在不损害提示对齐的前提下减少记忆现象。我们提出CAPTAIN，一种无需训练的框架，通过在去噪过程中直接修改潜在特征来缓解记忆。该方法首先采用基于频率的噪声初始化，降低早期去噪阶段复制记忆模式的倾向；随后识别最优去噪时间步与记忆区域；最后将非记忆参考图像中的语义对齐特征注入局部潜在空间，有效抑制记忆行为，同时保持提示一致性与图像质量。实验表明，与基于CFG的基线相比，CAPTAIN在显著降低记忆现象的同时，仍能维持强提示对齐能力。
原文摘要 · Abstract (English)
Diffusion models can unintentionally reproduce training examples, raising privacy and copyright concerns as these systems are increasingly deployed at scale. Existing inference-time mitigation methods typically manipulate classifier-free guidance (CFG) or perturb prompt embeddings; however, they often struggle to reduce memorization without compromising alignment with the conditioning prompt. We introduce CAPTAIN, a training-free framework that mitigates memorization by directly modifying latent features during denoising. CAPTAIN first applies frequency-based noise initialization to reduce the tendency to replicate memorized patterns early in the denoising process. It then identifies the optimal denoising timesteps for feature injection and localizes memorized regions. Finally, CAPTAIN injects semantically aligned features from non-memorized reference images into localized latent regions, suppressing memorization while preserving prompt fidelity and visual quality. Our experiments show that CAPTAIN achieves substantial reductions in memorization compared to CFG-based baselines while maintaining strong alignment with the intended prompt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。