解决多模态异构客户端的模型个性化问题，提升隐私保护下的协作效率。
Co-LoRA: Collaborative Model Personalization on Heterogeneous Multi-Modal Clients
- 设计任务相关性感知聚合策略，减少异构数据下的参数干扰。
- 提出维度不变的Co-LoRA模块，实现不同架构间的知识共享。
- 构建涵盖40个任务的多模态基准，覆盖时序分布偏移场景。
随着AI日益个性化（如智能体AI），对模型进行个性化的需求不断增长。个性化联邦学习（PFL）可在不泄露隐私的前提下，让各客户端协作利用其他客户端的知识以更好适应目标任务。然而现有PFL方法仍局限于数据与模型一致的简化场景。为迈向真实应用，本文突破上述限制，同时处理数据与模型异构性。提出一种任务相关性感知的模型聚合策略，降低异构数据下的参数干扰；并引入Co-LoRA，一种维度不变的模块，支持跨异构架构的知识共享。为模拟现实任务多样性，构建了一个涵盖40个不同任务、具有时间分布偏移的多模态PFL基准。大量实验表明，所提方法在异构场景下显著优于当前最先进的PFL方法。
原文摘要 · Abstract (English)
As AI becomes more personal, e.g., Agentic AI, there is an increasing need for personalizing models for various use cases. Personalized federated learning (PFL) enables each client to collaboratively leverage other clients' knowledge for better adaptation to the task of interest, without privacy risks. Despite its potential, existing PFL methods remain confined to rather simplified scenarios where data and models are the same across clients. To move towards realistic scenarios, we move beyond these restrictive assumptions by addressing both data and model heterogeneity. We propose a task-relevance-aware model aggregation strategy to reduce parameter interference under heterogeneous data. Moreover, we introduce Co-LoRA, a dimension-invariant module that enables knowledge sharing across heterogeneous architectures. To mimic the real-world task diversity, we propose a multi-modal PFL benchmark spanning 40 distinct tasks with distribution shifts over time. Extensive experiments shows that our proposed method significantly outperforms the state-of-the-art PFL methods under heterogeneous scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。