BLOSSOM让多模态联邦学习在数据缺失时仍能高效协作，提升模型性能。
BLOSSOM: Block-wise Federated Learning Over Shared and Sparse Observed Modalities
- 按模块分块聚合，共享通用部分，保留任务特有部分
- 模态缺失时性能比整体聚合高18.7%，独占模态下提升达37.7%
- 适合医疗、自动驾驶等多源异构数据场景
多模态联邦学习对自动驾驶和医疗等实际应用至关重要，但现有方法通常假设各客户端模态齐全，限制了实用性。本文提出BLOSSOM，一种面向共享与稀疏观测模态的无任务依赖多模态联邦学习框架。该框架支持任意模态子集的客户端，并允许灵活共享模型组件。为应对客户端与任务异构性，提出分块聚合策略：仅聚合共享组件，保留任务特定模块私有，实现部分个性化。在多个多样化多模态数据集上评估BLOSSOM，并分析模态缺失与个性化的影响。结果表明，分块个性化显著提升性能，尤其在严重模态稀疏场景下表现突出：在模态不完整情况下，相比全模型聚合平均提升18.7%；在模态独占设置中，提升高达37.7%，凸显分块学习对实际多模态联邦系统的重要性。
原文摘要 · Abstract (English)
Multimodal federated learning (FL) is essential for real-world applications such as autonomous systems and healthcare, where data is distributed across heterogeneous clients with varying and often missing modalities. However, most existing FL approaches assume uniform modality availability, limiting their applicability in practice. We introduce BLOSSOM, a task-agnostic framework for multimodal FL designed to operate under shared and sparsely observed modality conditions. BLOSSOM supports clients with arbitrary modality subsets and enables flexible sharing of model components. To address client and task heterogeneity, we propose a block-wise aggregation strategy that selectively aggregates shared components while keeping task-specific blocks private, enabling partial personalization. We evaluate BLOSSOM on multiple diverse multimodal datasets and analyse the effects of missing modalities and personalization. Our results show that block-wise personalization significantly improves performance, particularly in settings with severe modality sparsity. In modality-incomplete scenarios, BLOSSOM achieves an average performance gain of 18.7% over full-model aggregation, while in modality-exclusive settings the gain increases to 37.7%, highlighting the importance of block-wise learning for practical multimodal FL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。