通过生成特定领域提示,提升联邦学习在未知领域的泛化能力。
Federated Domain Generalization with Domain-specific Soft Prompts Generation
- 为每个领域设计专属软提示,融合内容与领域知识生成。
- 在未见领域上生成提示,推理性能超越现有方法。
- 适合需要跨域泛化的联邦学习场景,尤其关注未知领域表现。
提示学习已成为将CLIP适配下游任务的高效范式。相比传统微调,提示学习仅优化少量参数却能取得优异效果,特别适用于计算资源受限的联邦学习场景。然而,客户端间存在的领域偏移给下游任务适应带来挑战。现有基于提示学习的联邦领域泛化(FDG)方法通常从训练样本中学习软提示,以替代人工设计提示,增强联邦模型的泛化能力。但这些学习到的提示多样性不足,且易忽略未知领域信息。本文提出一种新颖有效的生成式方法——联邦领域泛化中的领域特定软提示生成(FedDSPG)。具体而言,在训练阶段,为每个领域引入领域特定软提示(DSPs),并将内容与领域知识整合进客户端间的生成模型。推理阶段,利用生成器为未见目标领域生成DSPs,从而指导未知领域下的下游任务。在多个公开数据集上的综合评估表明,本方法在联邦领域泛化任务中显著优于现有强基线,达到当前最优水平。
原文摘要 · Abstract (English)
Prompt learning has become an efficient paradigm for adapting CLIP to downstream tasks. Compared with traditional fine-tuning, prompt learning optimizes a few parameters yet yields highly competitive results, especially appealing in federated learning for computational efficiency. engendering domain shift among clients and posing a formidable challenge for downstream-task adaptation. Existing federated domain generalization (FDG) methods based on prompt learning typically learn soft prompts from training samples, replacing manually designed prompts to enhance the generalization ability of federated models. However, these learned prompts exhibit limited diversity and tend to ignore information from unknown domains. We propose a novel and effective method from a generative perspective for handling FDG tasks, namely federated domain generalization with domain-specific soft prompts generation (FedDSPG). Specifically, during training, we introduce domain-specific soft prompts (DSPs) for each domain and integrate content and domain knowledge into the generative model among clients. In the inference phase, the generator is utilized to obtain DSPs for unseen target domains, thus guiding downstream tasks in unknown domains. Comprehensive evaluations across several public datasets confirm that our method outperforms existing strong baselines in FDG, achieving state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。