用生成模型自建私密训练数据,性能不降反提升隐私保护。
Your Image Generator Is Your New Private Dataset
- 通过动态图文生成+高效微调,自动构建适配分类任务的合成数据集。
- 仅用合成数据训练的模型在10个基准上准确率媲美甚至超过真实数据训练。
- 合成数据显著降低成员推断攻击风险,隐私保护能力大幅提升。
生成式扩散模型已成为合成训练数据的强大工具,可缓解数据稀缺问题并降低下游监督学习的标注成本。然而,有效利用文本条件图像生成构建分类器训练集需解决三大挑战:设计有信息量的文本提示、将生成模型适配特定领域、确保鲁棒性能。本文提出文本条件知识回收(TCKR)流程,融合动态图像描述、参数高效扩散模型微调和生成式知识蒸馏技术,构建面向图像分类的定制化合成数据集。该流程在十个不同图像分类基准上进行了严格评估。结果表明,仅使用TCKR生成数据训练的模型,分类准确率与使用真实图像训练的模型相当,甚至在多个任务上更优。此外,评估显示这些合成数据训练的模型展现出显著增强的隐私特性:其对成员推断攻击的脆弱性大幅降低,平均成员推断AUC下降5.49点,证明了性能与隐私权衡的重大改进。研究发现,高质量合成数据可有效替代真实数据训练分类器,在保持优异性能的同时,实现卓越的隐私保护,这是一项有价值的涌现特性。代码与训练模型已开源。
原文摘要 · Abstract (English)
Generative diffusion models have emerged as powerful tools to synthetically produce training data, offering potential solutions to data scarcity and reducing labelling costs for downstream supervised deep learning applications. However, effectively leveraging text-conditioned image generation for building classifier training sets requires addressing key issues: constructing informative textual prompts, adapting generative models to specific domains, and ensuring robust performance. This paper proposes the Text-Conditioned Knowledge Recycling (TCKR) pipeline to tackle these challenges. TCKR combines dynamic image captioning, parameter-efficient diffusion model fine-tuning, and Generative Knowledge Distillation techniques to create synthetic datasets tailored for image classification. The pipeline is rigorously evaluated on ten diverse image classification benchmarks. The results demonstrate that models trained solely on TCKR-generated data achieve classification accuracies on par with (and in several cases exceeding) models trained on real images. Furthermore, the evaluation reveals that these synthetic-data-trained models exhibit substantially enhanced privacy characteristics: their vulnerability to Membership Inference Attacks is significantly reduced, with the membership inference AUC lowered by 5.49 points on average compared to using real training data, demonstrating a substantial improvement in the performance-privacy trade-off. These findings indicate that high-fidelity synthetic data can effectively replace real data for training classifiers, yielding strong performance whilst simultaneously providing improved privacy protection as a valuable emergent property. The code and trained models are available in the accompanying open-source repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。