提出VPDR方法,让联邦学习更私密且不丢精度。
Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning

- 根据特征维度方差自适应加噪,保关键信息
- 动态调整裁剪阈值,提升模型表现10%以上
- 适合注重隐私的医疗、金融等个性化场景
基于原型的个性化联邦学习(ProtoPFL)通过传输紧凑类别原型实现多领域高效适配,但直接共享原型存在隐私风险。常见防御方法是在原型计算前对每个样本进行ℓ₂裁剪以限制敏感度,再添加各向同性高斯噪声以满足局部差分隐私(LDP)。然而,各向同性高斯原型扰动(IGPP)通常过度扰动可区分维度,难以在裁剪阈值与表征保真度间取得平衡。本文提出客户端侧隐私插件VPDR,可无缝集成至现有ProtoPFL框架。受维度级类方差反映判别性的启发,我们引入方差自适应原型扰动(VPP),对判别性强的子空间分配较少噪声,从而保留语义可分性并保障隐私。进一步提出蒸馏引导裁剪正则化(DCR),使特征范数自适应聚焦于预设裁剪阈值,同时保持预测一致性。理论分析表明,在相同隐私约束下,我们的组内机制提供不低于各向同性基线的隐私保证。在多个多领域基准上的大量实验显示,VPDR在个性化联邦微调中实现了更优的隐私-效用权衡,性能优于IGPP,且不牺牲对真实攻击的鲁棒性。
原文摘要 · Abstract (English)
Prototype-based Personalized Federated Learning (ProtoPFL) enables efficient multi-domain adaptation by communicating compact class prototypes, but directly sharing them poses privacy risks. A common defense involves per-example $\ell_2$ clipping before prototype computation to bound sensitivity, followed by isotropic Gaussian noise to enforce Local Differential Privacy (LDP). However, Isotropic Gaussian Prototype Perturbation (IGPP) typically over-perturbs discriminative dimensions and struggles to balance the clipping threshold with representation fidelity. In this paper, we propose VPDR, a client-side privacy plug-in that seamlessly integrates into existing ProtoPFLs. Motivated by the observation that dimension-wise class variance reflects discriminability, we introduce Variance-adaptive Prototype Perturbation (VPP), which allocates less noise to discriminative subspaces, preserving semantic separability while ensuring privacy. We further develop Distillation-guided Clipping Regularization (DCR), which enables feature norms to adaptively concentrate near the predefined clipping threshold while maintaining prediction consistency. Theoretical analysis shows that our groupwise mechanism provides privacy guarantees no weaker than the isotropic baseline under the same privacy constraints. Extensive experiments on multi-domain benchmarks demonstrate that VPDR achieves a superior privacy-utility trade-off, outperforming IGPP in personalized federated fine-tuning without sacrificing robustness against realistic attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。