arXiv:2411.07889cs.LG2024-11

提出兼顾隐私与公平的分布式学习算法,支持强隐私保护下的高公平性模型训练。

A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data

  • 基于随机优化框架实现跨数据孤岛的记录级差分隐私
  • 在非凸条件下证明收敛性,突破以往需强凸假设的限制
  • 适用于医疗等敏感场景,适合关注隐私公平性的研究者

机器学习模型常在分散于不同机构(如医院)的敏感数据上训练,这些数据涉及医疗记录、种族/性别等。此类联邦学习模型可能用于重要决策(如医疗资源分配),面临两大挑战:一是保护个人数据隐私,即使其他机构或中心服务器被攻破也无法推断;二是确保决策对不同群体(如种族/性别)公平。本文提出一种新型私有且公平的联邦学习算法,满足跨孤岛记录级差分隐私(ISRL-DP),即每个孤岛发送的消息均满足记录级隐私保护。该框架可支持多种公平性准则,包括人口均等和等机会。我们在损失函数满足温和光滑性假设下证明了算法收敛性,而先前工作需强凸性才能保证收敛。作为分析副产品,首次给出了ISRL-DP非凸-强凹极小极大联邦学习的收敛性保障。实验表明,该算法在不同隐私水平下实现了当前最优的公平性-准确率权衡。

原文摘要 · Abstract (English)

Machine learning models are often trained on sensitive data (e.g., medical records and race/gender) that is distributed across different "silos" (e.g., hospitals). These federated learning models may then be used to make consequential decisions, such as allocating healthcare resources. Two key challenges emerge in this setting: (i) maintaining the privacy of each person's data, even if other silos or an adversary with access to the central server tries to infer this data; (ii) ensuring that decisions are fair to different demographic groups (e.g., race/gender). In this paper, we develop a novel algorithm for private and fair federated learning (FL). Our algorithm satisfies inter-silo record-level differential privacy (ISRL-DP), a strong notion of private FL requiring that silo i's sent messages satisfy record-level differential privacy for all i. Our framework can be used to promote different fairness notions, including demographic parity and equalized odds. We prove that our algorithm converges under mild smoothness assumptions on the loss function, whereas prior work required strong convexity for convergence. As a byproduct of our analysis, we obtain the first convergence guarantee for ISRL-DP nonconvex-strongly concave min-max FL. Experiments demonstrate the state-of-the-art fairness-accuracy tradeoffs of our algorithm across different privacy levels.

联邦学习隐私保护公平性差分隐私

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。