arXiv:2506.22374cs.LGcs.AI2025-06被引 1

用层叠理论提升多模态设备协作,让不同能力的终端更聪明地协同工作。

Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems

  • 基于层叠理论构建跨设备多模态协作框架,自动捕捉数据关联性。
  • 在毫米波波束成形和链路遮挡预测中,性能优于传统联邦学习方法。
  • 适合异构无线网络中多传感器、能力不一的边缘设备使用。

在大规模通信系统中,复杂场景需要边缘设备协同处理多种模态感知数据,以更全面理解环境并提高决策准确性。然而,传统联邦学习(FL)通常仅处理单模态数据,要求统一模型结构,无法充分利用多模态数据中的丰富信息,限制了其在多样模态与异构客户端环境中的应用。为此,我们提出 Sheaf-DMFL,一种基于层叠理论的去中心化多模态学习框架,增强具有不同模态的设备间的协作能力。每个客户端为不同模态配置局部特征编码器,输出拼接后输入任务特定层;同一模态的编码器在客户端间联合训练,而任务特定层间的内在关联通过层叠结构建模。为进一步提升学习能力,提出 Sheaf-DMFL-Att 算法,在客户端内部引入注意力机制,捕捉多模态间相关性。对 Sheaf-DMFL-Att 提供严格的收敛性分析,建立理论保证。在真实世界链路遮挡预测与毫米波波束成形场景中进行大量仿真,验证所提算法在异构无线通信系统中的优越性。

原文摘要 · Abstract (English)

In large-scale communication systems, increasingly complex scenarios require more intelligent collaboration among edge devices collecting various multimodal sensory data to achieve a more comprehensive understanding of the environment and improve decision-making accuracy. However, conventional federated learning (FL) algorithms typically consider unimodal datasets, require identical model architectures, and fail to leverage the rich information embedded in multimodal data, limiting their applicability to real-world scenarios with diverse modalities and varying client capabilities. To address this issue, we propose Sheaf-DMFL, a novel decentralized multimodal learning framework leveraging sheaf theory to enhance collaboration among devices with diverse modalities. Specifically, each client has a set of local feature encoders for its different modalities, whose outputs are concatenated before passing through a task-specific layer. While encoders for the same modality are trained collaboratively across clients, we capture the intrinsic correlations among clients' task-specific layers using a sheaf-based structure. To further enhance learning capability, we propose an enhanced algorithm named Sheaf-DMFL-Att, which tailors the attention mechanism within each client to capture correlations among different modalities. A rigorous convergence analysis of Sheaf-DMFL-Att is provided, establishing its theoretical guarantees. Extensive simulations are conducted on real-world link blockage prediction and mmWave beamforming scenarios, demonstrate the superiority of the proposed algorithms in such heterogeneous wireless communication systems.

多模态学习联邦学习无线通信层叠理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。