arXiv:2602.08590cs.LGcs.DB2026-02被引 1

解决联邦提示学习中本地与全局知识冲突问题

SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learning

  • 客户端自适应长度提示,全局保持固定长度便于聚合
  • 通过子空间精炼和发散控制提升跨设备性能
  • 适合数据/资源异构的隐私保护场景,提升模型鲁棒性

视觉语言预训练模型具备强泛化能力,但在隐私敏感的多方协作场景中,联邦优化面临通信开销高、客户端数据有限的挑战。联邦提示学习通过冻结视觉语言预训练模型(VLPM)主干,仅协同训练轻量级提示参数缓解该问题。然而,现有方法通常在客户端强制统一提示结构和长度,难以应对实际中数据分布与系统资源的异构性,可能加剧全局共享与本地最优知识之间的冲突。为此,我们提出SDFed框架,通过子空间精炼与发散控制,弥合本地-全局差异。SDFed维持固定长度的全局提示以实现高效聚合,同时允许各客户端学习可变长度的本地提示,更好匹配自身数据特性与计算能力。为缓解本地-全局冲突并促进有效知识迁移,SDFed引入局部提示的子空间精炼方法,以及信息保留与发散控制策略,在保留关键本地信息的同时,保持全局与局部表示间的合理可分性。多个数据集上的大量实验表明,SDFed在异构联邦设置下持续提升性能与鲁棒性。

原文摘要 · Abstract (English)

Vision-language pretrained models offer strong transferable representations, yet adapting them in privacy-sensitive multi-party settings is challenging due to the high communication cost of federated optimization and the limited local data on clients. Federated prompt learning mitigates this issue by keeping the VLPM backbone frozen and collaboratively training lightweight prompt parameters. However, existing approaches typically enforce a unified prompt structure and length across clients, which is inadequate under practical client heterogeneity in both data distributions and system resources, and may further introduce conflicts between globally shared and locally optimal knowledge. To address these challenges, we propose \textbf{SDFed}, a heterogeneous federated prompt learning framework that bridges Local-Global Discrepancy via Subspace Refinement and Divergence Control. SDFed maintains a fixed-length global prompt for efficient aggregation while allowing each client to learn a variable-length local prompt to better match its data characteristics and capacity. To mitigate local-global conflicts and facilitate effective knowledge transfer, SDFed introduces a subspace refinement method for local prompts and an information retention and divergence control strategy that preserves key local information while maintaining appropriate separability between global and local representations. Extensive experiments on several datasets demonstrate that SDFed consistently improves performance and robustness in heterogeneous federated settings.

联邦学习提示学习异构性视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。