首次为重尾随机微分方程建立瑞尼差分隐私保障，降低维度依赖。
Rényi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincaré Inequalities
- 基于分数阶庞加莱不等式构建新隐私流计算框架。
- 实现首个重尾SDE的瑞尼差分隐私保证，维度依赖显著减弱。
- 适用于研究深度学习中重尾噪声机制的学者与工程师。
近年来，刻画学习算法的差分隐私（DP）成为重大挑战。同时，大量研究关注带有重尾噪声的随机梯度下降（SGD），既作为现代深度学习模型的建模工具，也用于提升性能。然而，多数DP界主要针对轻尾噪声，在该条件下已获得良好保障，但现有方法无法直接推广至重尾场景。近期首次实现了重尾SGD的(0,δ)-DP保障，无需梯度截断。尽管揭示了DP与重尾算法间的联系，这些结果仍对参数量高度敏感，且难以扩展至主流的瑞尼差分隐私（RDP）。本文提出通过新瑞尼流计算与已知的分数阶庞加莱不等式，首次为重尾随机微分方程及其离散化形式提供RDP保障。在满足分数阶庞加莱不等式假设下，所得隐私界对维度的依赖远弱于先前工作。
原文摘要 · Abstract (English)
Characterizing the differential privacy (DP) of learning algorithms has become a major challenge in recent years. In parallel, many studies suggested investigating the behavior of stochastic gradient descent (SGD) with heavy-tailed noise, both as a model for modern deep learning models and to improve their performance. However, most DP bounds focus on light-tailed noise, where satisfactory guarantees have been obtained but the proposed techniques do not directly extend to the heavy-tailed setting. Recently, the first DP guarantees for heavy-tailed SGD were obtained. These results provide $(0,δ)$-DP guarantees without requiring gradient clipping. Despite casting new light on the link between DP and heavy-tailed algorithms, these results have a strong dependence on the number of parameters and cannot be extended to other DP notions like the well-established Rényi differential privacy (RDP). In this work, we propose to address these limitations by deriving the first RDP guarantees for heavy-tailed SDEs, as well as their discretized counterparts. Our framework is based on new Rényi flow computations and the use of well-established fractional Poincaré inequalities. Under the assumption that such inequalities are satisfied, we obtain DP guarantees that have a much weaker dependence on the dimension compared to prior art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。