提出去中心化零样本联邦学习框架,实现高效隐私保护的跨设备模型适配。
Zero-Shot Decentralized Federated Learning
- 采用迭代提示共享机制,客户端自主优化并交换文本提示。
- 在9个数据集上表现优于或持平主流方法,通信开销降低118倍。
- 适合对隐私和可扩展性要求高的真实场景部署。
CLIP通过零样本学习实现了无需微调的任务泛化。尽管提示技术如CoOp和CoCoOp提升了CLIP的适应性,但在联邦学习(FL)中的有效性仍是未解挑战。现有联邦提示学习方法如FedCoOp和FedTPG虽提升性能,却面临泛化不足、通信成本高及依赖中心服务器的问题,限制了可扩展性和隐私保护。我们提出零样本去中心化联邦学习(ZeroDFL),一种完全去中心化的框架,使分布式客户端无需中心协调即可实现零样本适应。ZeroDFL采用迭代提示共享机制,允许客户端优化并交换文本提示,显著提升泛化能力的同时大幅降低通信开销。我们在九个不同图像分类数据集上验证了ZeroDFL,结果表明其性能持续优于或等同于当前最佳联邦提示学习方法。更重要的是,该方法在完全去中心化设置下,相比FedTPG将通信开销减少118倍。这些成果表明,我们的方法不仅增强了联邦零样本学习的泛化能力,还提升了可扩展性、效率与隐私保护,为大规模视觉-语言模型在现实应用中的去中心化适配铺平道路。
原文摘要 · Abstract (English)
CLIP has revolutionized zero-shot learning by enabling task generalization without fine-tuning. While prompting techniques like CoOp and CoCoOp enhance CLIP's adaptability, their effectiveness in Federated Learning (FL) remains an open challenge. Existing federated prompt learning approaches, such as FedCoOp and FedTPG, improve performance but face generalization issues, high communication costs, and reliance on a central server, limiting scalability and privacy. We propose Zero-shot Decentralized Federated Learning (ZeroDFL), a fully decentralized framework that enables zero-shot adaptation across distributed clients without a central coordinator. ZeroDFL employs an iterative prompt-sharing mechanism, allowing clients to optimize and exchange textual prompts to enhance generalization while drastically reducing communication overhead. We validate ZeroDFL on nine diverse image classification datasets, demonstrating that it consistently outperforms--or remains on par with--state-of-the-art federated prompt learning methods. More importantly, ZeroDFL achieves this performance in a fully decentralized setting while reducing communication overhead by 118x compared to FedTPG. These results highlight that our approach not only enhances generalization in federated zero-shot learning but also improves scalability, efficiency, and privacy preservation--paving the way for decentralized adaptation of large vision-language models in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。