arXiv:2607.19580cs.LGcs.CR2026-07

只对输入数据加隐私保护,训练更准且隐私更强。

End-to-End Differential Privacy in Training Deep Neural Network Classifiers

  • 用狄利克雷机制随机化softmax输出,实现输入端差分隐私
  • 在CIFAR10上ε=4时准确率达88.17%,比之前高近10个百分点
  • 适合标签公开场景,尤其适用于高隐私要求的医疗图像任务

差分隐私机器学习可在保护敏感数据的同时训练模型,但现有方法常对输入和标签同时加密,当标签可公开时过于保守。本文提出新框架:仅对训练输入加隐私保护,标签保持公开。针对带有softmax输出层的神经网络,将输入到softmax输出的映射视为单位单纯形上的变换,通过应用狄利克雷机制对输出随机化,实现端到端的输入隐私保护。由于数据在多轮训练中重复使用,采用黎尼(Rényi)差分隐私理论,对狄利克雷机制在多次使用下的隐私强度给出紧致边界。实验表明,在CIFAR10、MNIST、MedMNIST、FashionMNIST和SVHN上从零开始训练时,本方法在所有隐私预算下均达到新最优准确率。特别地,在(ε, δ)-差分隐私设置下δ=10⁻⁵时,于CIFAR10上ε=4的准确率从78.37%提升至88.17%;即使ε=1时仍达82.96%,显著优于先前工作。

原文摘要 · Abstract (English)

Differentially private machine learning enables model training on sensitive data while ensuring that individual data is unlikely to be recoverable from the parameters of the resulting model. However, existing work often privatizes both training inputs and their labels, and these protections may be conservative when labels are public or can be safely made public. Therefore, in this work we propose a novel private training framework that instead privatizes training inputs while keeping labels public. We consider neural networks with softmax output layers, and thus the mapping from training inputs to the output of the softmax layer is a mapping onto the unit simplex. We randomize softmax outputs during training by applying the Dirichlet mechanism to enforce differential privacy for the training inputs, hence the ``end-to-end'' label. Because training data is reused across multiple training epochs, we use the notion of \Renyi differential privacy to formulate tight bounds on the strength of privacy provided by the Dirichlet mechanism across repeated uses. We show empirically that we attain new state-of-the-art accuracy when training from scratch on CIFAR10, MNIST, MedMNIST, FashionMNIST, and SVHN across all privacy budgets evaluated. Notably, when implementing $(ε, δ)$-differential privacy with $δ=10^{-5}$, we improve the prior state-of-the-art accuracy from $78.37\%$ to $88.17\%$ at $ε=4$ on CIFAR10, and our approach has $82.96\%$ accuracy even for $ε=1$, which significantly outperforms prior work.

差分隐私神经网络隐私保护图像分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。