提出有界自适应裁剪，提升差分隐私学习中少数群体的预测准确率。
Mitigating Disparate Impact of Differentially Private Learning through Bounded Adaptive Clipping
- 引入可调下界防止梯度过度抑制，缓解自适应裁剪对少数群体的负面影响。
- 在偏斜和时尚MNIST上，最差类准确率提升超10个百分点。
- 适合关注隐私保护下模型公平性的研究者与工程实践者。
差分隐私（DP）已成为隐私保护机器学习的重要框架。然而，现有DP学习方法常对模型预测产生差异性影响，尤其对少数群体。梯度裁剪常用于DP学习，会抑制难样本的大梯度。我们发现，自适应裁剪会将裁剪边界缩小至极小值以匹配多数群体的拟合情况，从而显著降低其他群体的准确率。为此，我们提出有界自适应裁剪，引入可调下界以防止过度梯度抑制。相比无界自适应裁剪，该方法在偏斜和时尚MNIST数据集上使最差类准确率提升超过10个百分点；相比自动裁剪提升7个百分点，相比恒定裁剪提升5个百分点。代码已开源：https://github.com/TrustworthyMLHelsinki/adaptive-clipping-fairness。
原文摘要 · Abstract (English)
Differential privacy (DP) has become an essential framework for privacy-preserving machine learning. Existing DP learning methods, however, often have disparate impacts on model predictions, e.g., for minority groups. Gradient clipping, which is often used in DP learning, can suppress larger gradients from challenging samples. We show that this problem is amplified by adaptive clipping, which will often shrink the clipping bound to tiny values to match a well-fitting majority, while significantly reducing the accuracy for others. We propose bounded adaptive clipping, which introduces a tunable lower bound to prevent excessive gradient suppression. Our method improves worst-class accuracy by over 10 percentage points on Skewed and Fashion MNIST compared to unbounded adaptive clipping, 7 points compared to Automatic clipping, and 5 points compared to constant clipping. The code is available at https://github.com/TrustworthyMLHelsinki/adaptive-clipping-fairness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。