arXiv:2505.22703cs.LGcs.CR2025-05

在隐私保护下实现公平学习的速率约束优化,提升模型公平性与隐私安全。

Private Rate-Constrained Optimization with Applications to Fair Learning

  • 基于拉格朗日对偶的随机梯度上升下降算法,解决群体公平性约束难题
  • 每轮仅需私密统计小批量数据的分布直方图,隐私开销可控
  • 在神经网络上同时保障隐私、性能与公平性,优于现有方法

许多可信机器学习问题可表述为对子群体预测率的约束,包括群体公平性约束(如人口均等、几率均等)。本文研究在差分隐私(DP)下的此类约束优化问题。标准的隐私优化方法如DP-SGD依赖于可分解到单个样本的目标函数,支持逐样本梯度裁剪与噪声添加。但速率约束依赖于各群体的聚合统计量,导致样本间存在依赖关系,破坏了可分解性。为此,我们提出RaCO-DP,一种改进的随机梯度下降-上升(SGDA)算法,用于求解速率约束问题的拉格朗日形式。通过精心设计,引入约束带来的额外隐私成本被限制为在每步中对每个小批量私密估计直方图的成本。我们通过新颖的SGDA收敛分析证明算法有效性,该分析利用了对偶参数的线性结构。实验表明,本方法在群体公平性约束下,帕累托优于现有私有学习方法,并在神经网络上实现了优异的隐私-效用-公平性平衡。

原文摘要 · Abstract (English)

Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.). In this work, we study such constrained minimization problems under differential privacy (DP). Standard DP optimization techniques like DP-SGD rely on objectives that decompose over individual examples, enabling per-example gradient clipping and noise addition. Rate constraints, however, depend on aggregate statistics across groups, creating inter-sample dependencies that violate this decomposability. To address this, we develop RaCO-DP, a DP variant of Stochastic Gradient Descent-Ascent (SGDA) that solves the Lagrangian formulation of rate constraint problems. Through careful design, the extra privacy cost incurred by incorporating these constraints in our approach is limited to that of privately estimating a histogram over each mini-batch at every step. We prove the convergence of our algorithm through a novel analysis of SGDA that leverages the linear structure of the dual parameter. Empirical results show that our method Pareto-dominates existing private learning approaches under group fairness constraints and also achieves strong privacy-utility-fairness performance on neural networks.

差分隐私公平学习约束优化神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。