发现持续学习模型易受跨阶段对抗攻击，存在严重安全风险
Exploring Cross-Stage Adversarial Transferability in Class-Incremental Continual Learning
- 用早期模型生成的对抗样本攻击后期模型，验证跨阶段攻击可行性
- 多数持续学习方法在后期对攻击的防御能力显著下降，鲁棒性持续退化
- 现有对抗训练防御手段对跨阶段攻击效果有限，需新防御策略
类别增量持续学习通过使分类模型在学习新类别时保留旧知识来应对灾难性遗忘。然而，该过程中的模型对对抗攻击的脆弱性尚未得到充分研究。本文首次探索了跨阶段对抗迁移性：即使用前期模型生成的对抗样本攻击后期模型。研究发现，持续学习方法对此类攻击高度敏感，引发严重安全问题。我们通过分析各阶段模型相似性及鲁棒性渐进退化解释该现象。此外，现有基于对抗训练的防御方法对跨阶段攻击仍不充分有效。代码已开源于 https://github.com/mcml-official/CSAT。
原文摘要 · Abstract (English)
Class-incremental continual learning addresses catastrophic forgetting by enabling classification models to preserve knowledge of previously learned classes while acquiring new ones. However, the vulnerability of the models against adversarial attacks during this process has not been investigated sufficiently. In this paper, we present the first exploration of vulnerability to stage-transferred attacks, i.e., an adversarial example generated using the model in an earlier stage is used to attack the model in a later stage. Our findings reveal that continual learning methods are highly susceptible to these attacks, raising a serious security issue. We explain this phenomenon through model similarity between stages and gradual robustness degradation. Additionally, we find that existing adversarial training-based defense methods are not sufficiently effective to stage-transferred attacks. Codes are available at https://github.com/mcml-official/CSAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。