发现分类模型输出标签会泄露隐私，提出新方法提升差分隐私持续学习的保护效果。
Privacy Leakage via Output Label Space and Differentially Private Continual Learning
- 将分类模型输出标签空间视为隐私侧信道，提出针对性防御
- 在Split-CIFAR-100和Split-ImageNet-R上实现更高准确率与更强隐私保障
- 适用于需持续学习且重视数据隐私的场景，如医疗或金融建模
差分隐私（DP）是一种正式的隐私保护框架，可在训练机器学习（ML）模型时保护个人数据。已有研究指出，即使模型训练本身具备差分隐私，其作为更大系统的一部分，仍可能通过所谓的隐私侧信道泄露信息。本文识别出分类模型的输出标签空间即为一类隐私侧信道，并提出了一个具体的隐私攻击方案。该侧信道在持续学习（CL）中尤为显著，因输出标签空间随时间动态变化。为此，本文提出面向持续学习的差分隐私形式化定义，明确了与现有方法的差异。进一步提出两种消除该侧信道的方法：对敏感数据中的标签应用最优差分隐私机制，以及使用大规模公共标签空间。通过适配预训练模型评估了这两种方法的权衡。实验表明，在Split-CIFAR-100和Split-ImageNet-R数据集上，所提模型在差分隐私条件下始终优于先前工作，且保持更高的准确率。
原文摘要 · Abstract (English)
Differential privacy (DP) is a formal privacy framework that enables training machine learning (ML) models while protecting individuals' data. As pointed out by prior work, ML models are part of larger systems, which can lead to so-called privacy side-channels even if the model training itself is DP. We identify the output label space of a classification model as such a privacy side-channel and show a concrete privacy attack that exploits it. The side-channel becomes highly relevant in continual learning (CL), where the output label space changes over time. To reason about privacy guarantees in CL, we introduce a formalisation of DP for CL, which also clarifies how our approach differs from existing approaches. We propose and evaluate two methods for eliminating this side-channel: applying an optimal DP mechanism to release the labels in the sensitive data, and using a large public label space. We explore the trade-offs of these methods through adapting pre-trained models. We demonstrate empirically that our models consistently achieve higher accuracy under DP than previous work over both Split-CIFAR-100 and Split-ImageNet-R, with a stronger privacy model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。