提出概念级选择性学习,防止模型记忆敏感特征。
Beyond Memorization: Selective Learning for Copyright-Safe Diffusion Model Training
- 训练时通过梯度投影剔除敏感特征的更新信号。
- 在不丢弃数据前提下,大幅降低模型记忆风险。
- 适合需版权安全的生成式AI开发与部署场景。
大规模文本到图像扩散模型中的记忆化现象带来显著的安全与知识产权风险,可能导致恶意属性提取及敏感或专有特征的未经授权再现。传统去记忆化技术如正则化和数据过滤虽能缓解对特定训练样本的过拟合,却无法系统性防止禁止概念级别的特征内化。简单删除含敏感特征的所有图像会浪费大量训练数据,因此亟需一种概念级别的选择性学习方法。本文提出一种梯度投影方法,旨在强制实现概念级特征排除。该防御机制在反向传播过程中运行,通过系统识别并移除与禁止属性嵌入对齐的训练信号。具体而言,我们将每个梯度更新投影到敏感特征嵌入空间的正交补空间,从而使其对模型权重的影响归零。该方法可无缝集成至标准扩散模型训练流程,并与现有防御措施互补。我们针对试图提取特征的对手进行了分析。在广泛实验中,结果表明本框架在严格保护生成质量与语义保真度的同时,显著降低了记忆化程度。通过将记忆控制重构为选择性学习,本方法为知识产权安全与隐私保护的生成式AI建立了新范式。
原文摘要 · Abstract (English)
Memorization in large-scale text-to-image diffusion models poses significant security and intellectual property risks, enabling adversarial attribute extraction and the unauthorized reproduction of sensitive or proprietary features. While conventional dememorization techniques, such as regularization and data filtering, limit overfitting to specific training examples, they fail to systematically prevent the internalization of prohibited concept-level features. Simply discarding all images containing a sensitive feature wastes invaluable training data, necessitating a method for selective learning at the concept level. We introduce a gradient projection method designed to enforce a stringent requirement of concept-level feature exclusion. Our defense operates during backpropagation by systematically identifying and excising training signals aligned with embeddings of prohibited attributes. Specifically, we project each gradient update onto the orthogonal complement of the sensitive feature's embedding space, thereby zeroing out its influence on the model's weights. Our method integrates seamlessly into standard diffusion model training pipelines and complements existing defenses. We analyze our method against an adversary aiming for feature extraction. In extensive experiments, we demonstrate that our framework drastically reduces memorization while rigorously preserving generation quality and semantic fidelity. By reframing memorization control as selective learning, our approach establishes a new paradigm for IP-safe and privacy-preserving generative AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。