扩散模型训练中存在偏差泛化现象,可能泄露隐私。
Biased Generalization in Diffusion Models
- 通过对比不同数据集生成样本与训练数据的距离,量化模型偏好
- 测试损失下降时,生成样本反而更接近训练数据,出现偏差
- 适合关注隐私保护与模型安全的开发者阅读
生成模型的泛化能力指从有限数据集中学习底层分布并生成新样本的能力,通常以保留数据上的表现和样本质量评估。实践中常在测试损失最小时停止训练,视作泛化良好的指标。我们提出质疑:训练过程中存在一个偏倚泛化阶段,此时测试损失持续下降,但模型更倾向于生成与训练数据异常接近的样本。通过在同一网络上训练两组互不重叠的数据集,并比较生成样本间的距离及其与训练数据的相似性,我们提出了偏差的定量度量,并在真实图像上验证了该现象的存在。进一步利用可控的分层数据模型,结合精确得分与真实统计信息,精确刻画了偏倚的出现机制。我们将其归因于深度网络中特征学习的顺序性:粗粒度结构早期以数据无关方式学习,而细粒度特征后期逐渐依赖具体训练样本。结果表明,在标准泛化准则下最优的早停策略,对隐私敏感场景可能不足。
原文摘要 · Abstract (English)
Generalization in generative modeling is defined as the ability to learn an underlying distribution from a finite dataset and produce novel samples, with evaluation largely driven by held-out performance and perceived sample quality. In practice, training is often stopped at the minimum of the test loss, taken as an operational indicator of generalization. We challenge this viewpoint by identifying a phase of biased generalization during training, in which the model continues to decrease the test loss while favoring samples with anomalously high proximity to training data. By training the same network on two disjoint datasets and comparing the mutual distances of generated samples and their similarity to training data, we introduce a quantitative measure of bias and demonstrate its presence on real images. We then study the mechanism of bias, using a controlled hierarchical data model where access to exact scores and ground-truth statistics allows us to precisely characterize its onset. We attribute this phenomenon to the sequential nature of feature learning in deep networks, where coarse structure is learned early in a data-independent manner, while finer features are resolved later in a way that increasingly depends on individual training samples. Our results show that early stopping at the test loss minimum, while optimal under standard generalization criteria, may be insufficient for privacy-critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。