隐私增强在特定条件下反而提升模型泛化能力，打破传统认知。
Unveiling the Non-Monotonic Effect of Privacy on Generalization under Byzantine Robustness
- 区分高噪声与低噪声隐私场景，揭示隐私与泛化关系非单调
- 强隐私下提升隐私可降低泛化误差，弱隐私下则相反
- 理论结合实验，解释分布式学习中隐私与鲁棒性的复杂权衡
近期研究确立了分布式学习中拜占庭鲁棒性、局部差分隐私（LDP）和优化误差之间的基本三难困境。本文表明，这一三难困境并不普遍适用于泛化误差，其表现关键取决于隐私机制。具体而言，在高噪声场景（强隐私）下，我们证明增加隐私会降低泛化误差，即鲁棒性与隐私间无冲突；而在低噪声场景（弱隐私）下，鲁棒性与隐私的矛盾重新出现，隐私提升反而损害泛化性能。我们的理论通过在LDP约束下对拜占庭鲁棒分布式学习算法稳定性的上下界匹配，解释了这种反直觉的非单调现象。实证评估验证并深化了这些理论发现。
原文摘要 · Abstract (English)
Recent work has established a fundamental trilemma between Byzantine robustness, local differential privacy (LDP), and optimization error in distributed learning. We show that this trilemma does not universally extend to generalization error, but instead depends critically on the privacy regime. Specifically, in the high-noise regime (strong privacy), we prove that increasing privacy reduces the generalization error, i.e., there is no tension between robustness and privacy. In the low-noise regime (weaker privacy), however, the tension between robustness and privacy reappears and increasing privacy indeed degrades generalization. Our theory explains this surprising non-monotonic behavior of the generalization error via matching lower and upper bounds on the algorithmic stability of Byzantine-robust distributed learning under LDP constraints. We corroborate and further analyze these theoretical findings with empirical evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。