11种前沿隐私保护机器学习方法实测，发现部分方法在新条件下失效。
Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)
- 对11种顶级差分隐私机器学习方法进行可复现性实验
- 部分方法在原条件外性能显著下降，真实性存疑
- 提出应对隐私噪声等挑战的可复现实践建议
近期大量研究提出新型差分隐私机器学习（DPML）技术，声称达到新的最先进（SoTA）性能，并以实证结果为证。然而，目前尚无共识判断哪些方法真正有效，或其宣称是否属实。由于代码库、数据集、方法和模型架构的异质性，不同方法间难以直接比较。本文对近期文献中的11种主流DPML技术开展可复现性与可重复性（R+R）实验。结果显示：部分方法经得起检验，但另一些在初始实验条件之外表现不佳。我们还探讨了DPML复现中的独特挑战，如差分隐私噪声带来的额外随机性，并提出应对策略。最终总结出获得科学可靠结果的最佳实践。
原文摘要 · Abstract (English)
There is a flurry of recent research papers proposing novel differentially private machine learning (DPML) techniques. These papers claim to achieve new state-of-the-art (SoTA) results and offer empirical results as validation. However, there is no consensus on which techniques are most effective or if they genuinely meet their stated claims. Complicating matters, heterogeneity in codebases, datasets, methodologies, and model architectures make direct comparisons of different approaches challenging. In this paper, we conduct a reproducibility and replicability (R+R) experiment on 11 different SoTA DPML techniques from the recent research literature. Results of our investigation are varied: while some methods stand up to scrutiny, others falter when tested outside their initial experimental conditions. We also discuss challenges unique to the reproducibility of DPML, including additional randomness due to DP noise, and how to address them. Finally, we derive insights and best practices to obtain scientifically valid and reliable results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。