arXiv:2411.03752cs.LGcs.CR2024-11AAAI

通过让模型损失函数曲率变大,实现隐蔽的后门攻击。

Deferred Poisoning: Making the Model More Vulnerable via Hessian Singularization

  • 利用海森矩阵奇异化,在训练时保持正常表现。
  • 测试时对微小扰动极度敏感,准确率大幅下降。
  • 适合研究模型鲁棒性与安全防御的学者参考。

近期研究表明深度学习模型极易遭受投毒攻击,但传统攻击常因训练与验证集表现不一致而暴露。本文提出一种更隐蔽的延迟投毒攻击(Deferred Poisoning Attack),使模型在训练和验证阶段表现正常,却对逃避攻击或自然噪声极为敏感。该方法通过确保每个样本的损失值与正常训练模型相近,同时引入大局部曲率,实现高隐蔽性。具体通过提出的奇异化正则化项,使最优解处海森矩阵奇异,从而导致微小扰动即引发损失显著上升,造成性能严重下降。理论与实证分析均验证了该方法的有效性,实验基于图像分类任务,且在自然噪声下也证实了其危害性,为安全研究提供了新视角。

原文摘要 · Abstract (English)

Recent studies have shown that deep learning models are very vulnerable to poisoning attacks. Many defense methods have been proposed to address this issue. However, traditional poisoning attacks are not as threatening as commonly believed. This is because they often cause differences in how the model performs on the training set compared to the validation set. Such inconsistency can alert defenders that their data has been poisoned, allowing them to take the necessary defensive actions. In this paper, we introduce a more threatening type of poisoning attack called the Deferred Poisoning Attack. This new attack allows the model to function normally during the training and validation phases but makes it very sensitive to evasion attacks or even natural noise. We achieve this by ensuring the poisoned model's loss function has a similar value as a normally trained model at each input sample but with a large local curvature. A similar model loss ensures that there is no obvious inconsistency between the training and validation accuracy, demonstrating high stealthiness. On the other hand, the large curvature implies that a small perturbation may cause a significant increase in model loss, leading to substantial performance degradation, which reflects a worse robustness. We fulfill this purpose by making the model have singular Hessian information at the optimal point via our proposed Singularization Regularization term. We have conducted both theoretical and empirical analyses of the proposed method and validated its effectiveness through experiments on image classification tasks. Furthermore, we have confirmed the hazards of this form of poisoning attack under more general scenarios using natural noise, offering a new perspective for research in the field of security.

投毒攻击模型鲁棒性安全防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。