通过分布统计检测,破解了蒸馏扩散模型的训练数据溯源难题。
Distributional Statistics Restore Training Data Auditability in One-step Distilled Diffusion Models
- 利用核方法衡量输出分布差异,追踪模型训练源头
- 即使无单张图像记忆,仍可识别训练数据占比不足10%的非法数据
- 适用于实时生成模型,为可信生成式AI提供审计工具
基于网络规模、来源不确定的图像数据训练的扩散模型泛滥,使得判断模型是否未经授权学习了特定版权数据变得至关重要,却长期未获解决。现有方法依赖模型对训练图像的重构能力优于未见图像的过拟合效应,但该效应在蒸馏过程中消失——当前主流部署方式将计算密集的教师模型压缩为高效的一步生成学生模型,学生仅从教师生成的输出中训练,从未直接接触原始训练数据,导致无法通过单例记忆追溯上游数据,形成模型洗白漏洞。本文揭示:尽管不存在单张图像记忆,但学生输出分布仍更接近教师训练分布而非其他非训练数据分布,即存在分布级记忆链。基于此,我们提出一种基于核方法的分布不一致检测器,通过比较候选数据集与学生输出分布的统计一致性,判断其是否比外部非训练数据集更匹配,从而回溯训练来源。跨多个基准和蒸馏设置的评估表明,即使非法数据在候选集中占比极低(如低于10%),仍能可靠检测,确立了分布级审计作为应对模型洗白的对策,推动可问责生成式AI生态的发展。
原文摘要 · Abstract (English)
The proliferation of diffusion models trained on web-scale, provenance-uncertain image collections has made it essential, yet technically unresolved, to determine whether a model has learned from specific copyrighted data without authorization. Current methods primarily rely on the memorization effect, whereby models reconstruct their training images better than unseen ones, to detect unauthorized training data on a per-instance basis. This effect, however, vanishes under distillation, the now-dominant deployment pipeline that compresses compute-intensive teacher diffusion models into efficient {\em student one-step generators} mimicking the teacher's output for real-time user access. As the students train exclusively on teacher-generated outputs and never directly see the teacher's original training data, they carry no per-instance memorization of that upstream data, creating a model laundering loophole that severs the auditable link between a deployed model and its upstream training data. We nonetheless reveal that a distributional memory chain survives under distillation: the student's output distribution remains closer to the teacher's training distribution than to any non-training reference, even if no single training instance is memorized. Exploiting this chain, we develop a distributional unauthorized training data detector, grounded in kernel-based distribution discrepancy, that determines if a candidate dataset of unknown composition is statistically aligned with the student-generated distribution more than held-out non-training datasets, thus tracing provenance back to the teacher's training data. Evaluation across benchmarks and distillation setups confirms reliable detection even when unauthorized data forms a minority of the candidate set, establishing distribution-level auditing as a countermeasure to model laundering and a paradigm for accountable generative AI ecosystems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。