通过流形几何优化隐私保护,让生成模型在强隐私下仍保持高精度。
StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers

- 利用噪声到数据的直线路径几何特性,分阶段分配隐私预算。
- ε=1时,仅释放类条件均值即达0.76准确率,FID为237。
- 适用于需要强隐私保障的文本条件生成任务,如图像合成。
文本条件生成模型的差分隐私训练在强隐私设置下会出现性能断崖。本文从修正流(rectified flow)的几何结构出发:从噪声到数据的直线插值路径上,贝叶斯最优速度在噪声端主要由少数类条件矩决定,而越接近数据端则越依赖样本特异性结构。StraightDP从头到尾利用这种异质性:少量预算用于一次性释放去相关后的类条件矩,可将其蒸馏进权重或采样时注入;其余预算则通过预定义的DP-SGD用于数据端,超越矩的范围。在MNIST上ε=1时,仅释放的矩即实现0.76下游准确率,生成样本具原型特征,FID为237;统一使用DP-SGD仅达0.21。基于释放机制构建的流水线在公开潜在空间中实现0.81准确率与FID 56。约束多模态主干的每标记流范数,在不改变预训练损失的前提下,显著提升极端噪声像素空间下的下游准确率,且随着隐私增强,效果持续改善。该释放的矩还可迁移到冻结的SD3-medium模型中,采样时注入优于DP-LoRA训练,仅用极少预算。
原文摘要 · Abstract (English)
Differentially private (DP) training of text-conditioned generative models suffers a utility cliff at strong privacy. We revisit this problem through the geometry of rectified flows: along the straight interpolation between noise and data, the Bayes-optimal velocity is governed to leading order at the noise end by a few class-conditional moments, and increasingly sample-specific structure matters toward the data end. StraightDP exploits this heterogeneity end to end. A small budget share releases whitened class-conditional moments once, to be distilled into the weights or injected at sampling time. The rest is spent by pre-declared DP-SGD toward the data end, beyond the moments' reach. At $\varepsilon=1$ on MNIST, the released moments alone already attain $0.76$ downstream accuracy with prototype-like samples and an FID of $237$, and uniform DP-SGD attains $0.21$. The pipeline built on the release reaches $0.81$ accuracy at FID $56$ in a public latent space. Constraining per-token stream norms of the multimodal backbone leaves the pretraining loss unchanged yet improves downstream accuracy in the extreme-noise pixel-space regime, and its accuracy effect becomes monotonically more favorable as privacy strengthens. The released moments also port to frozen SD3-medium, where sampling-time injection beats DP-LoRA training at a fraction of the budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。