arXiv:2503.22263cs.LGcs.CV2025-03NeurIPS

提出FLIP框架,系统评估联邦提示学习性能

FLIP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learning

  • 构建多场景评估框架,测试8种联邦提示学习方法
  • 在12个数据集上验证其低资源消耗下仍具强泛化能力
  • 适合关注隐私保护与小样本学习的研究者

随着对隐私和数据安全的日益重视,联邦学习作为一种去中心化训练机器学习模型的方法得到广泛应用,无需共享原始数据。提示学习通过微调预训练模型的提示嵌入,在联邦设置中具有显著优势,可降低计算成本和通信开销,同时利用视觉语言模型(如CLIP)的强大性能与泛化能力。本文聚焦联邦学习与提示学习的交叉领域,尤其针对视觉语言模型,提出一个全面的评估框架FLIP,用于评估8种前沿联邦提示学习算法。该框架在4种联邦学习协议、12个公开数据集上,覆盖6种不同评估场景。结果表明,提示学习在分布内和分布外设置下均保持良好泛化性能,且资源消耗极低。本工作凸显了联邦提示学习在数据稀缺、未见类别及跨域分布偏移环境中的有效性。所有实现算法的代码已开源,以促进该领域的进一步研究。

原文摘要 · Abstract (English)

The increasing emphasis on privacy and data security has driven the adoption of federated learning, a decentralized approach to train machine learning models without sharing raw data. Prompt learning, which fine-tunes prompt embeddings of pretrained models, offers significant advantages in federated settings by reducing computational costs and communication overheads while leveraging the strong performance and generalization capabilities of vision-language models such as CLIP. This paper addresses the intersection of federated learning and prompt learning, particularly for vision-language models. In this work, we introduce a comprehensive framework, named FLIP, to evaluate federated prompt learning algorithms. FLIP assesses the performance of 8 state-of-the-art federated prompt learning methods across 4 federated learning protocols and 12 open datasets, considering 6 distinct evaluation scenarios. Our findings demonstrate that prompt learning maintains strong generalization performance in both in-distribution and out-of-distribution settings with minimal resource consumption. This work highlights the effectiveness of federated prompt learning in environments characterized by data scarcity, unseen classes, and cross-domain distributional shifts. We open-source the code for all implemented algorithms in FLIP to facilitate further research in this domain.

联邦学习提示学习视觉语言模型评估框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。