提出一种无需服务器的多模态联邦学习框架,解决异构设备间信息不对齐问题。
PID-Guided Partial Alignment for Multimodal Decentralized Federated Learning
- 将特征分解为冗余、独有、协同三部分,按模态条件通信
- 在多个基准上优于现有方法,且每条链路传输量可控
- 适合异构设备协作,尤其适用于无中心节点的分布式场景
多模态去中心化联邦学习(DFL)需支持持有不同模态子集和模型组件的代理间协作,且在无协调服务器或全局网络视图的点对点(P2P)拓扑中运行。传统多模态训练依赖单一共享表示,隐含假设异构代理可在相同通信链路上交换并聚合相同模型组件,但在多模态DFL中该假设失效:单模态与多模态代理可能通过共享拓扑推送不兼容更新,削弱跨代理迁移与跨模态交互。本文提出无服务器框架PARSE,引入部分信息分解(PID),使每个代理将潜在特征划分为冗余、唯一和协同三类(“特征分裂”),并在模态条件化的P2P拓扑上进行分片感知通信。训练中,代理仅交换与其邻居语义可对齐的特征分片,依据其共享的模态与模型组件(“部分对齐”)。该设计避免集中调度与梯度手术式冲突处理,同时兼容标准DFL约束及多种P2P拓扑结构。在多个基准与异构代理混合设置下,PARSE持续优于任务、模态与混合共享型多模态DFL基线,且每链接负载保持受限。融合策略与分割比例的消融实验,结合定性特征分析与拓扑研究,验证了所提分片感知设计的鲁棒性与通信效率。
原文摘要 · Abstract (English)
Multimodal decentralized federated learning (DFL) must support collaboration among agents that hold different modality subsets and often different model components, while operating over peer-to-peer (P2P) overlays without a coordinating server or a global network view. A key obstacle is that conventional multimodal training often relies on a single shared representation, which implicitly assumes that heterogeneous peers can exchange and aggregate the same model components over the same communication links. In multimodal DFL, this assumption breaks down: uni- and multimodal agents may push incompatible updates through shared overlays, weakening both inter-agent transfer and cross-modal interaction. We present PARSE, a server-free framework that brings partial information decomposition (PID) into multimodal DFL. Each agent splits its latent features into redundant, unique, and synergistic slices ("feature fission"), and performs slice-aware communication over modality-conditioned P2P overlays. During training, agents exchange only the slices that are semantically alignable with their neighbors, according to the modalities and model components they share ("partial alignment"). This design avoids centralized orchestration and gradient-surgery style conflict handling, while remaining compatible with standard DFL constraints and a range of P2P overlay topologies. Across multiple benchmarks and heterogeneous peer mixes, PARSE consistently outperforms task-, modality-, and hybrid-sharing multimodal DFL baselines while keeping per-link payloads bounded. Ablations on fusion choices and split ratios, together with qualitative feature analyses and overlay-topology studies, demonstrate the robustness and communication efficiency of the proposed slice-aware design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。