arXiv:2502.18706cs.LGcs.CR2025-02ICLR被引 19

动态分配隐私预算,让模型训练后期更精准

Differentially Private Federated Learning With Time-Adaptive Privacy Spending

  • 按训练阶段动态调整各客户端的隐私预算
  • 实验显示在相同隐私下准确率提升5%以上
  • 适合对隐私敏感且需高精度的场景

联邦学习结合差分隐私(DP)可实现协作建模的同时严格保护隐私。传统方法假设所有客户端在各轮训练中均匀消耗隐私预算,但研究表明:早期训练依赖粗粒度特征,对信噪比要求较低;后期则需细粒度特征,受益于更高信噪比。本文提出时间自适应的差分隐私联邦学习框架,允许客户端在时间与个体间非均匀地使用隐私预算。通过在早期保留预算,后期可增加噪声抑制,从而提升对精细特征的学习能力。理论证明,隐私约束更严格的客户端若采用非均匀支出策略,其模型效用优于预算宽松者。标准数据集上的实验验证了该策略在隐私-效用权衡上的优势,相较基线方案显著提升性能。

原文摘要 · Abstract (English)

Federated learning (FL) with differential privacy (DP) provides a framework for collaborative machine learning, enabling clients to train a shared model while adhering to strict privacy constraints. The framework allows each client to have an individual privacy guarantee, e.g., by adding different amounts of noise to each client's model updates. One underlying assumption is that all clients spend their privacy budgets uniformly over time (learning rounds). However, it has been shown in the literature that learning in early rounds typically focuses on more coarse-grained features that can be learned at lower signal-to-noise ratios while later rounds learn fine-grained features that benefit from higher signal-to-noise ratios. Building on this intuition, we propose a time-adaptive DP-FL framework that expends the privacy budget non-uniformly across both time and clients. Our framework enables each client to save privacy budget in early rounds so as to be able to spend more in later rounds when additional accuracy is beneficial in learning more fine-grained features. We theoretically prove utility improvements in the case that clients with stricter privacy budgets spend budgets unevenly across rounds, compared to clients with more relaxed budgets, who have sufficient budgets to distribute their spend more evenly. Our practical experiments on standard benchmark datasets support our theoretical results and show that, in practice, our algorithms improve the privacy-utility trade-offs compared to baseline schemes.

联邦学习差分隐私隐私预算自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。