arXiv:2505.18770cs.CVcs.LG2025-05

提出新方法提升提示生成稳定性,让模型更可靠地泛化到未知领域。

Dual-Path Stable Soft Prompt Generation for Domain Generalization

  • 双路径架构:主生成器结合负样本提示,抑制误导信息。
  • 在5个基准数据集上超越现有方法,且提示输出随随机种子变化小。
  • 适合关注模型鲁棒性和跨域泛化的研究者使用。

领域泛化(DG)旨在利用一个或多个相关但不同的源域数据训练模型,使其能有效泛化到未见的分布外目标域。受大型预训练视觉语言模型成功启发,提示调优已成为一种有效的泛化策略。然而,传统方法依赖人工或固定提示输入,难以捕捉领域特定特征。近期动态生成实例与领域特定提示的方法虽有所改进,但存在显著问题:相同输入在不同随机种子下生成的提示差异大且效果差,我们称之为提示可变性。为此,本文引入负学习机制,提出双路径稳定软提示生成(DPSPG),一种基于Transformer的框架,以提升提示的稳定性和泛化能力。具体而言,DPSPG通过互补提示生成器生成负提示,降低引入误导信息的风险。理论与实证分析表明,负学习通过增大有效边界并减小梯度范数上界,提升了提示的鲁棒性与有效性。在五个DG基准数据集上的大量实验显示,DPSPG持续优于当前最优方法,同时保持提示稳定性。

原文摘要 · Abstract (English)

Domain generalization (DG) aims to learn a model using data from one or multiple related but distinct source domains that can generalize well to unseen out-of-distribution target domains. Inspired by the success of large pre-trained vision-language models (VLMs), prompt tuning has emerged as an effective generalization strategy. However, it often struggles to capture domain-specific features due to its reliance on manually or fixed prompt inputs. Recently, some prompt generation methods have addressed this limitation by dynamically generating instance-specific and domain-specific prompts for each input, enriching domain information and demonstrating potential for enhanced generalization. Through further investigation, we identify a notable issue in existing prompt generation methods: the same input often yields significantly different and suboptimal prompts across different random seeds, a phenomenon we term Prompt Variability. To address this, we introduce negative learning into the prompt generation process and propose Dual-Path Stable Soft Prompt Generation (DPSPG), a transformer-based framework designed to improve both the stability and generalization of prompts. Specifically, DPSPG incorporates a complementary prompt generator to produce negative prompts, thereby reducing the risk of introducing misleading information. Both theoretical and empirical analyses demonstrate that negative learning leads to more robust and effective prompts by increasing the effective margin and reducing the upper bound of the gradient norm. Extensive experiments on five DG benchmark datasets show that DPSPG consistently outperforms state-of-the-art methods while maintaining prompt stability.

领域泛化提示生成稳定性视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。