微调会牺牲模型鲁棒性，需权衡准确率与抗干扰能力。
On the Robustness Tradeoff in Fine-Tuning
- 通过对比7种微调策略，发现准确率与鲁棒性存在系统性权衡。
- 在简单任务上，仅更新参数偏置（BitFit）性能超平均75%以上。
- 复杂任务中，信息密集层微调（如Compacter）更优，提升达57.5%。
微调已成为将预训练模型适配到下游任务的标准方法，但其对模型鲁棒性的影响尚不明确。本文系统评估了6个基准数据集上7种不同微调策略的鲁棒性与准确率表现。结果表明，对抗鲁棒性与准确率之间存在一致的权衡关系。对于简单任务，仅更新偏置项（BitFit）效果显著，在CIFAR-10和CIFAR-100上的性能优于平均值超过75%。而在复杂任务上，如通过Compacter微调注意力层等信息密集模块，可实现更优的帕累托前沿：在Caltech-256和CUB-200上分别领先平均值57.5%和34.6%。此外，模型对分布外数据的鲁棒性与准确率高度相关。研究强调了在实际部署中引入鲁棒性感知微调的重要性。
原文摘要 · Abstract (English)
Fine-tuning has become the standard practice for adapting pre-trained models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-accuracy trade-off in fine-tuning. We evaluate the robustness and accuracy of fine-tuned models over 6 benchmark datasets and 7 different fine-tuning strategies. We observe a consistent trade-off between adversarial robustness and accuracy. Peripheral updates such as BitFit are more effective for simple tasks -- over 75% above the average measured by the area under the Pareto frontiers on CIFAR-10 and CIFAR-100. In contrast, fine-tuning information-heavy layers, such as attention layers via Compacter, achieves a better Pareto frontier on more complex tasks -- 57.5% and 34.6% above the average on Caltech-256 and CUB-200, respectively. Lastly, we observe that the robustness of fine-tuning against out-of-distribution data closely tracks accuracy. These insights emphasize the need for robustness-aware fine-tuning to ensure reliable real-world deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。