研究测试时自适应的现实数据投毒攻击，发现现有方法高估了其风险。
On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning
- 提出无需良性数据的现实化投毒方法，更贴近真实攻击场景。
- 实验证明现有TFA方法比预想更鲁棒，性能下降有限。
- 设计防御策略，助力构建抗攻击的测试时自适应模型。
测试时自适应（TTA）在推理阶段利用测试数据更新模型权重以提升泛化能力，但由此引入了对抗风险。已有研究显示，使用精心构造的对抗性测试样本（即测试时投毒数据）会导致良性样本性能下降。然而，若投毒假设过于理想化，这种风险可能被夸大。本文首先梳理了测试时数据投毒的现实假设，包括白盒与灰盒攻击、是否可访问良性数据、攻击顺序等。随后，提出一种无需良性数据即可生成有效投毒样本的方法，并设计了合理的分布内攻击目标。此外，还构建了两种针对TTA的攻击目标。对现有攻击方法的基准测试表明，当前的TTA方法比先前认为的更具鲁棒性。同时，分析了有效的防御策略，为开发对抗鲁棒的TTA方法提供指导。代码已开源：https://github.com/Gorilla-Lab-SCUT/RTTDP。
原文摘要 · Abstract (English)
Test-time adaptation (TTA) updates the model weights during the inference stage using testing data to enhance generalization. However, this practice exposes TTA to adversarial risks. Existing studies have shown that when TTA is updated with crafted adversarial test samples, also known as test-time poisoned data, the performance on benign samples can deteriorate. Nonetheless, the perceived adversarial risk may be overstated if the poisoned data is generated under overly strong assumptions. In this work, we first review realistic assumptions for test-time data poisoning, including white-box versus grey-box attacks, access to benign data, attack order, and more. We then propose an effective and realistic attack method that better produces poisoned samples without access to benign samples, and derive an effective in-distribution attack objective. We also design two TTA-aware attack objectives. Our benchmarks of existing attack methods reveal that the TTA methods are more robust than previously believed. In addition, we analyze effective defense strategies to help develop adversarially robust TTA methods. The source code is available at https://github.com/Gorilla-Lab-SCUT/RTTDP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。