arXiv:2506.06823cs.CVcs.AI2025-06

探索视觉提示在鲁棒模型中的继承能力,提出新方法提升泛化性。

Exploring Visual Prompting: Robustness Inheritance and Beyond

  • 首次研究鲁棒源模型下视觉提示的性能表现
  • 提出提示边界松弛策略,显著提升泛化能力
  • 轻量级插件式设计,适配多种下游任务

视觉提示(Visual Prompting, VP)是一种高效的迁移学习方法,在视觉任务中展现出潜力。然而,以往研究仅关注标准源模型下的VP表现,尚未明确其在鲁棒源模型下的效果:鲁棒性能否成功继承?该过程是否仍存在鲁棒性与泛化能力之间的权衡?若存在,是否有针对VP的专门缓解策略?本文首次系统回答这三个问题,并给出肯定答案。为缓解此权衡,我们提出提示边界松弛(Prompt Boundary Loosening, PBL)策略。该策略轻量、可即插即用,与VP天然兼容,可在源模型为鲁棒模型时有效实现鲁棒性的成功继承,同时显著提升VP在多个下游数据集上的泛化能力。大量实验表明,该发现具有普遍性,所提策略带来显著收益。

原文摘要 · Abstract (English)

Visual Prompting (VP), an efficient method for transfer learning, has shown its potential in vision tasks. However, previous works focus exclusively on VP from standard source models, it is still unknown how it performs under the scenario of a robust source model: Can the robustness of the source model be successfully inherited? Does VP also encounter the same trade-off between robustness and generalization ability as the source model during this process? If such a trade-off exists, is there a strategy specifically tailored to VP to mitigate this limitation? In this paper, we thoroughly explore these three questions for the first time and provide affirmative answers to them. To mitigate the trade-off faced by VP, we propose a strategy called Prompt Boundary Loosening (PBL). As a lightweight, plug-and-play strategy naturally compatible with VP, PBL effectively ensures the successful inheritance of robustness when the source model is a robust model, while significantly enhancing VP's generalization ability across various downstream datasets. Extensive experiments across various datasets show that our findings are universal and demonstrate the significant benefits of the proposed strategy.

视觉提示鲁棒性迁移学习泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。