解决多用户共享数据的隐私训练问题，提出高效且实用的隐私保护算法。
It's My Data Too: Private ML for Datasets with Multi-User Training Examples
- 定义多归属场景下的用户级差分隐私，明确每个用户贡献上限。
- 设计贪心算法解决贡献上限问题，在多个任务中表现稳定。
- 揭示贡献限制中的偏差-方差权衡，对实际应用有指导意义。
我们针对用户级差分隐私（DP）下的模型训练展开研究，其中每个训练样本可能归属于多个用户，即多归属模型。我们首先提出了在多归属模型下合适的用户级差分隐私定义。多归属模型下的训练依赖于解决贡献约束问题——即选择一个子集，使得每位用户关联的样本数量受限。我们提出一种贪心基线算法来解决该问题，并在合成逻辑回归任务和Transformer训练任务上进行实验，评估了多种基于不同技术与标准优化选取子集的变体算法。结果表明，基线算法在多数情况下仍具竞争力，同时深化了对贡献约束解法中固有偏差-方差权衡的实践理解。
原文摘要 · Abstract (English)
We initiate a study of algorithms for model training with user-level differential privacy (DP), where each example may be attributed to multiple users, which we call the multi-attribution model. We first provide a carefully chosen definition of user-level DP under the multi-attribution model. Training in the multi-attribution model is facilitated by solving the contribution bounding problem, i.e. the problem of selecting a subset of the dataset for which each user is associated with a limited number of examples. We propose a greedy baseline algorithm for the contribution bounding problem. We then empirically study this algorithm for a synthetic logistic regression task and a transformer training task, including studying variants of this baseline algorithm that optimize the subset chosen using different techniques and criteria. We find that the baseline algorithm remains competitive with its variants in most settings, and build a better understanding of the practical importance of a bias-variance tradeoff inherent in solutions to the contribution bounding problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。