无需重新训练，即可生成满足任意隐私要求的模型。
Differentially Private Model Merging
- 通过随机选择与线性组合后处理已有模型。
- 在私有均值估计中精确刻画了隐私与效用权衡。
- 适合需要灵活调整隐私级别的部署场景。
在机器学习中，推理或部署时的隐私要求常因政策、法规或用户偏好变化而调整。本文旨在基于一组在同一数据集上训练、具备不同隐私/效用权衡的已有模型，构造出可满足任意目标差分隐私（DP）要求的最终模型，且无需额外训练。提出两种后处理技术：随机选择与线性组合，以生成满足任意目标隐私参数的私有模型。从R'enyi DP和隐私损失分布的角度，对一般问题及私有均值估计问题提供了隐私会计分析，精确刻画了隐私/效用权衡，并对比了两种机制。实验证明了方法的有效性，在多个模型及合成与真实世界数据集上验证了分析结果。
原文摘要 · Abstract (English)
In machine learning, privacy requirements at inference or deployment time often evolve due to changing policies, regulations, or user preferences. In this work, we aim to construct a magnitude of models to satisfy any target differential privacy (DP) requirement without additional training, given a set of existing models trained on the same dataset with different privacy/utility tradeoffs. We propose two post-processing techniques, namely random selection and linear combination, to generate final private models satisfying any target privacy parameter. We provide privacy accounting of these approaches from the lens of R'enyi DP and privacy loss distributions on general problems, as well as on private mean estimation, where we precisely characterize the privacy/utility tradeoffs and compare the two mechanisms. Empirically, we demonstrate the effectiveness of our approaches and validate our analyses on several models and both synthetic and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。