对比四种参数高效微调方法在对抗攻击下的表现
Understanding Robustness of Parameter-Efficient Tuning for Image Classification
- 测试四种参数高效微调方法在对抗攻击下的稳定性
- 发现可学习参数量影响模型鲁棒性，且对抗样本具有迁移性
- 适合关注模型安全性的视觉任务研究者参考
参数高效微调（PET）通过冻结预训练模型并引入少量可学习参数来调整下游任务的预测结果。尽管已有多种PET方法提出，但其鲁棒性尚未被系统研究。本文系统评估了四种经典PET方法（如VPT、Adapter、AdaptFormer和LoRA）在白盒攻击和信息扰动下的表现。在白盒攻击中，采用FGSM和PGD攻击分析性能；进一步探究了对抗样本的迁移性及可学习参数数量对鲁棒性的影响。在信息扰动场景下，引入四种扰动策略：块级丢弃、像素级丢弃、块随机打乱和高斯噪声，全面评估这些PET方法在信息丢失情况下的鲁棒性。研究增强了对PET方法鲁棒性的理解，为计算机视觉应用中提升性能提供了重要参考。代码已开源：https://github.com/JCruan519/PETRobustness。
原文摘要 · Abstract (English)
Parameter-efficient tuning (PET) techniques calibrate the model's predictions on downstream tasks by freezing the pre-trained models and introducing a small number of learnable parameters. However, despite the numerous PET methods proposed, their robustness has not been thoroughly investigated. In this paper, we systematically explore the robustness of four classical PET techniques (e.g., VPT, Adapter, AdaptFormer, and LoRA) under both white-box attacks and information perturbations. For white-box attack scenarios, we first analyze the performance of PET techniques using FGSM and PGD attacks. Subsequently, we further explore the transferability of adversarial samples and the impact of learnable parameter quantities on the robustness of PET methods. Under information perturbation attacks, we introduce four distinct perturbation strategies, including Patch-wise Drop, Pixel-wise Drop, Patch Shuffle, and Gaussian Noise, to comprehensively assess the robustness of these PET techniques in the presence of information loss. Via these extensive studies, we enhance the understanding of the robustness of PET methods, providing valuable insights for improving their performance in computer vision applications. The code is available at https://github.com/JCruan519/PETRobustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。