单任务数据投毒可严重破坏持续学习模型的稳定与适应能力。
Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning
- 攻击者仅知当前任务，利用图像损坏实施投毒。
- 投毒使模型既难保持旧知识，又无法有效学新任务。
- 提出基于任务向量的检测方法，提升持续学习安全性。
本研究关注持续学习（CL）中被忽视的安全威胁——数据投毒。数据投毒指恶意篡改训练数据以影响模型预测，近期已被证实威胁到持续学习的训练稳定性。现有工作多聚焦于依赖特定场景的攻击，而本文关注更简单且现实的单任务投毒（STP）威胁：攻击者仅了解当前任务，无模型、历史或未来任务信息。研究发现，在此严格限制下，攻击者仍可通过标准图像损坏显著破坏模型性能。STP攻击能同时损害算法的稳定性（旧任务表现下降）和塑性（新任务适应能力减弱）。最后，提出一种高层防御框架，并设计基于任务向量的投毒任务检测方法。代码已开源：https://github.com/stapaw/STP.git。
原文摘要 · Abstract (English)
Our research addresses the overlooked security concerns related to data poisoning in continual learning (CL). Data poisoning - the intentional manipulation of training data to affect the predictions of machine learning models - was recently shown to be a threat to CL training stability. While existing literature predominantly addresses scenario-dependent attacks, we propose to focus on a more simple and realistic single-task poison (STP) threats. In contrast to previously proposed poisoning settings, in STP adversaries lack knowledge and access to the model, as well as to both previous and future tasks. During an attack, they only have access to the current task within the data stream. Our study demonstrates that even within these stringent conditions, adversaries can compromise model performance using standard image corruptions. We show that STP attacks are able to strongly disrupt the whole continual training process: decreasing both the stability (its performance on past tasks) and plasticity (capacity to adapt to new tasks) of the algorithm. Finally, we propose a high-level defense framework for CL along with a poison task detection method based on task vectors. The code is available at https://github.com/stapaw/STP.git .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。