通过分频域逐步生成图像,提升私有数据训练下的图像质量与隐私保护。
Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling
- 在小波域中分两阶段训练:先对低频特征加隐私保护,再用公开模型上采样细节。
- 在MS-COCO和MM-CelebA-HQ上生成图像质量优于现有主流DP方法。
- 适合关注隐私保护下图像生成质量的研究者与应用开发者。
在敏感图像数据集上训练的生成模型可能记忆并复现具体训练样本,因此强隐私保障至关重要。尽管差分隐私(DP)提供了理论框架,但标准的DP微调(如DP-SGD)常因对所有参数无差别加噪而导致图像质量严重下降,尤其在高频纹理部分。本文基于假设:图像中最具隐私敏感性的成分多为小波域中的低频部分(如面部特征、物体轮廓),而高频部分大多通用且公开。据此提出两阶段谱系DP图像生成框架:(1)对敏感图像的低分辨率小波系数,使用差分隐私微调自回归谱系图像分词器模型;(2)采用公开预训练超分辨率模型进行高分辨率上采样。通过将隐私预算限制在图像全局结构的第一阶段,并利用DP的后处理性质实现细节优化,实现了隐私与实用性的良好权衡。在MS-COCO和MM-CelebA-HQ数据集上的实验表明,该方法生成的图像在质量与风格捕捉上优于其他领先的DP图像生成框架。
原文摘要 · Abstract (English)
Generative models trained on sensitive image datasets risk memorizing and reproducing individual training examples, making strong privacy guarantees essential. While differential privacy (DP) provides a principled framework for such guarantees, standard DP finetuning (e.g., with DP-SGD) often results in severe degradation of image quality, particularly in high-frequency textures, due to the indiscriminate addition of noise across all model parameters. In this work, we propose a spectral DP framework based on the hypothesis that the most privacy-sensitive portions of an image are often low-frequency components in the wavelet space (e.g., facial features and object shapes) while high-frequency components are largely generic and public. Based on this hypothesis, we propose the following two-stage framework for DP image generation with coarse image intermediaries: (1) DP finetune an autoregressive spectral image tokenizer model on the low-resolution wavelet coefficients of the sensitive images, and (2) perform high-resolution upsampling using a publicly pretrained super-resolution model. By restricting the privacy budget to the global structures of the image in the first stage, and leveraging the post-processing property of DP for detail refinement, we achieve promising trade-offs between privacy and utility. Experiments on the MS-COCO and MM-CelebA-HQ datasets show that our method generates images with improved quality and style capture relative to other leading DP image frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。