用平滑替换硬截断，让隐私训练更公平
SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
- 用tanh函数替代硬截断，保留梯度相对大小
- 在多个数据集上使少数群体差异降低超87%
- 适合关注隐私与公平的医疗、金融等场景
差分隐私（DP）能有效保护敏感数据，但常导致模型性能下降和公平性受损，尤其对少数群体影响更大。其主要原因是DP-SGD中的梯度截断会过度抑制少数群体的学习信号。尽管自适应截断可提升效用，仍依赖统一的硬截断，可能限制公平性。为此，我们提出SoftAdaClip，一种新的差分隐私训练方法，用基于tanh的平滑变换替代硬截断，既能控制敏感度，又保持梯度相对大小。我们在MIMIC-III（临床文本）、GOSSIS-eICU（结构化医疗数据）和Adult Income（表格数据）等多个数据集上进行了评估。结果表明，SoftAdaClip相较于DP-SGD将子群体差异降低最多达87%，相较于Adaptive-DPSGD降低最多达48%，且差异具有统计显著性。这些发现强调了将平滑变换与自适应机制结合对于实现公平且私密的模型训练的重要性。
原文摘要 · Abstract (English)
Differential privacy (DP) provides strong protection for sensitive data, but often reduces model performance and fairness, especially for underrepresented groups. One major reason is gradient clipping in DP-SGD, which can disproportionately suppress learning signals for minority subpopulations. Although adaptive clipping can enhance utility, it still relies on uniform hard clipping, which may restrict fairness. To address this, we introduce SoftAdaClip, a differentially private training method that replaces hard clipping with a smooth, tanh-based transformation to preserve relative gradient magnitudes while bounding sensitivity. We evaluate SoftAdaClip on various datasets, including MIMIC-III (clinical text), GOSSIS-eICU (structured healthcare), and Adult Income (tabular data). Our results show that SoftAdaClip reduces subgroup disparities by up to 87% compared to DP-SGD and up to 48% compared to Adaptive-DPSGD, and these reductions in subgroup disparities are statistically significant. These findings underscore the importance of integrating smooth transformations with adaptive mechanisms to achieve fair and private model training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。