arXiv:2410.03782cs.LGcs.CV2024-10ICLR被引 22

无需训练,动态调整模型权重以提升鲁棒性。

DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation

  • 根据测试样本的熵动态计算插值系数,实现自适应权重融合。
  • 在14个视觉任务上显著提升性能,计算开销几乎不变。
  • 适合需要快速适配且无训练条件的下游应用。

在下游任务中微调预训练基础模型时,需在不重新训练整个模型的前提下保证对分布偏移的鲁棒性。尽管现有权重插值方法简单有效,但其静态特性限制了性能。本文提出DaWin,一种无需训练的动态权重插值方法,通过计算各模型在每个未标记测试样本上的熵来评估模型专长,并动态生成样本级插值系数。与依赖额外训练学习系数的方法不同,本方法完全免训。此外,提出一种混合建模策略,大幅降低动态插值带来的推理开销。我们在大规模视觉识别基准上验证了DaWin,涵盖14项任务(包括ImageNet及其五个分布偏移子集)和八项多任务分类任务。结果表明,DaWin在各类设置下均取得显著性能提升,且计算开销极低。我们进一步分析了其理论行为以解释其实验成功原因。

原文摘要 · Abstract (English)

Adapting a pre-trained foundation model on downstream tasks should ensure robustness against distribution shifts without the need to retrain the whole model. Although existing weight interpolation methods are simple yet effective, we argue that their static nature limits downstream performance while achieving efficiency. In this work, we propose DaWin, a training-free dynamic weight interpolation method that leverages the entropy of individual models over each unlabeled test sample to assess model expertise, and compute per-sample interpolation coefficients dynamically. Unlike previous works that typically rely on additional training to learn such coefficients, our approach requires no training. Then, we propose a mixture modeling approach that greatly reduces inference overhead raised by dynamic interpolation. We validate DaWin on the large-scale visual recognition benchmarks, spanning 14 tasks across robust fine-tuning -- ImageNet and derived five distribution shift benchmarks -- and multi-task learning with eight classification tasks. Results demonstrate that DaWin achieves significant performance gain in considered settings, with minimal computational overhead. We further discuss DaWin's analytic behavior to explain its empirical success.

模型融合动态权重鲁棒性免训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。