在保护隐私的前提下，让服务器图像生成模型与客户端任务模型共同提升。
FedMMKT:Co-Enhancing a Server Text-to-Image Model and Client Task Models in Multi-Modal Federated Learning
- 通过联邦学习融合多方多模态数据，实现模型协同优化。
- 无需集中数据即可提升图像生成与特定任务模型性能。
- 适合移动端、物联网等隐私敏感场景的多模态应用。
文本到图像（T2I）模型在众多应用中展现出强大能力。然而，由于隐私限制，获取特定任务所需的数据常受阻碍，导致T2I模型难以适配专业场景。与此同时，现代移动系统与物联网基础设施蕴藏着丰富的多模态数据资源。本文提出联邦多模态知识迁移（FedMMKT）框架，支持在不泄露数据隐私的前提下，利用去中心化的多模态数据，实现服务器端T2I模型与客户端任务特定模型的协同增强。
原文摘要 · Abstract (English)
Text-to-Image (T2I) models have demonstrated their versatility in a wide range of applications. However, adaptation of T2I models to specialized tasks is often limited by the availability of task-specific data due to privacy concerns. On the other hand, harnessing the power of rich multimodal data from modern mobile systems and IoT infrastructures presents a great opportunity. This paper introduces Federated Multi-modal Knowledge Transfer (FedMMKT), a novel framework that enables co-enhancement of a server T2I model and client task-specific models using decentralized multimodal data without compromising data privacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。