arXiv:2410.09125cs.LGcs.AI2024-10被引 3

通过维度变换与梯度归一化,防止标签泄露同时保持模型性能

Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation

  • 引入SecDT模块实现标签与扩展类别的双向映射,从源头减少泄露
  • 梯度归一化消除类别间差异,对抗方向、范数等攻击,提升隐私保护
  • 在多个真实数据集上显著降低攻击成功率,适合隐私敏感的联邦学习场景

两方分裂学习已成为垂直联邦学习的主流范式。为保护标签方隐私,分裂学习采用分割模型,仅交换输入和中间表示(IR)的梯度。然而,近期研究证明分裂学习仍易受标签推断攻击。尽管已有防御方法,但多数效果有限或严重损害模型性能。本文提出一种新型两方分裂学习方法,在保持高模型效用的同时抵御现有标签推断攻击。首先设计维度变换模块SecDT,实现原始标签与扩展K类标签间的双向映射,从方向上缓解标签泄露;其次提出梯度归一化算法,消除不同类别梯度的幅度差异;最后引入软最大归一化高斯噪声,抑制隐私泄露并使K对攻击者不可知。在真实数据集上实验,包括两个二分类数据集(Avazu、Criteo)和三个多分类数据集(MNIST、FashionMNIST、CIFAR-10),评估了方向、范数、谱分析及模型重构等当前主流攻击。结果表明,所提方法在多个攻击下均表现优异,例如在Avazu数据集上,四种主要攻击的平均AUC下降达0.4532±0.0127。

原文摘要 · Abstract (English)

Two-party split learning has emerged as a popular paradigm for vertical federated learning. To preserve the privacy of the label owner, split learning utilizes a split model, which only requires the exchange of intermediate representations (IRs) based on the inputs and gradients for each IR between two parties during the learning process. However, split learning has recently been proven to survive label inference attacks. Though several defense methods could be adopted, they either have limited defensive performance or significantly negatively impact the original mission. In this paper, we propose a novel two-party split learning method to defend against existing label inference attacks while maintaining the high utility of the learned models. Specifically, we first craft a dimension transformation module, SecDT, which could achieve bidirectional mapping between original labels and increased K-class labels to mitigate label leakage from the directional perspective. Then, a gradient normalization algorithm is designed to remove the magnitude divergence of gradients from different classes. We propose a softmax-normalized Gaussian noise to mitigate privacy leakage and make our K unknowable to adversaries. We conducted experiments on real-world datasets, including two binary-classification datasets (Avazu and Criteo) and three multi-classification datasets (MNIST, FashionMNIST, CIFAR-10); we also considered current attack schemes, including direction, norm, spectral, and model completion attacks. The detailed experiments demonstrate our proposed method's effectiveness and superiority over existing approaches. For instance, on the Avazu dataset, the attack AUC of evaluated four prominent attacks could be reduced by 0.4532+-0.0127.

分裂学习隐私保护标签泄露联邦学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。