利用训练中的随机性提升隐私保护，降低数据泄露风险。
Leveraging Randomness in Model and Data Partitioning for Privacy Amplification
- 通过随机划分数据和模型参数，让样本只参与部分训练。
- 新方法比传统采样方式隐私增益更强，可减少隐私泄露。
- 适合关注联邦学习中隐私保护的开发者和研究者。
我们研究训练过程中的固有随机性如何用于隐私放大：每个样本（或联邦学习中的客户端）仅参与随机选择的部分训练。这包括数据分区（样本仅参与部分训练迭代）和模型分区（样本仅更新部分模型参数）。我们将该框架应用于联邦学习中的模型并行，其中每个客户端随机更新子网络，以降低内存和计算开销。结果显示，现有方法如模型分割或丢弃（dropout）带来的隐私放大效果远超以往分析所捕捉。此外，我们提出平衡迭代子采样（Balanced Iteration Subsampling），确保每个样本参与固定次数的训练迭代，其隐私放大效果优于传统的泊松（i.i.d.）采样。结果表明，训练过程中的结构化随机性与数据复杂交互，可系统性地实现显著隐私放大。
原文摘要 · Abstract (English)
We study how inherent randomness in the training process -- where each sample (or client in federated learning) contributes only to a randomly selected portion of training -- can be leveraged for privacy amplification. This includes (1) data partitioning, where a sample participates in only a subset of training iterations, and (2) model partitioning, where a sample updates only a subset of the model parameters. We apply our framework to model parallelism in federated learning, where each client updates a randomly selected subnetwork to reduce memory and computational overhead, and show that existing methods, e.g. model splitting or dropout, provide a significant privacy amplification gain not captured by previous privacy analysis techniques. Additionally, we introduce Balanced Iteration Subsampling, a new data partitioning method where each sample (or client) participates in a fixed number of training iterations. We show that this method yields stronger privacy amplification than Poisson (i.i.d.) sampling of data (or clients). Our results demonstrate that randomness in the training process, which is structured rather than i.i.d. and interacts with data in complex ways, can be systematically leveraged for significant privacy amplification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。