arXiv:2508.11159cs.LG2025-08被引 4

解决物联网多模态数据量与质量不均衡问题,提升在线联邦学习效果

Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning

  • 提出并行的QQR算法,动态平衡多模态数据量与质量
  • 在真实数据集上验证,性能优于基准方法
  • 适合边缘计算中资源受限的多模态设备

物联网(IoT)生态系统从传感器、摄像头和麦克风等多元来源产生海量多模态数据。随着边缘智能的发展,物联网设备已从简单的数据采集单元演变为具备计算能力的节点,能够对异构多模态数据进行本地化处理。这一演进要求分布式学习范式能高效处理此类数据。同时,数据持续生成且边缘设备存储容量有限,需采用在线学习框架。多模态在线联邦学习(MMO-FL)因此成为满足这些需求的有前景方案。然而,由于物联网设备固有的不稳定性,数据收集过程中常出现模态数量与质量不平衡(QQI)现象,给MMO-FL带来新挑战。本文系统研究了QQI在MMO-FL框架中的影响,并进行了全面的理论分析,量化了两类不平衡如何降低学习性能。为应对这些挑战,我们提出一种基于原型学习的模态数量与质量重平衡(QQR)算法,可与训练过程并行运行。在两个真实世界多模态数据集上的大量实验表明,所提QQR算法在模态不平衡条件下始终优于基线方法,展现出优异的学习性能。

原文摘要 · Abstract (English)

The Internet of Things (IoT) ecosystem produces massive volumes of multimodal data from diverse sources, including sensors, cameras, and microphones. With advances in edge intelligence, IoT devices have evolved from simple data acquisition units into computationally capable nodes, enabling localized processing of heterogeneous multimodal data. This evolution necessitates distributed learning paradigms that can efficiently handle such data. Furthermore, the continuous nature of data generation and the limited storage capacity of edge devices demand an online learning framework. Multimodal Online Federated Learning (MMO-FL) has emerged as a promising approach to meet these requirements. However, MMO-FL faces new challenges due to the inherent instability of IoT devices, which often results in modality quantity and quality imbalance (QQI) during data collection. In this work, we systematically investigate the impact of QQI within the MMO-FL framework and present a comprehensive theoretical analysis quantifying how both types of imbalance degrade learning performance. To address these challenges, we propose the Modality Quantity and Quality Rebalanced (QQR) algorithm, a prototype learning based method designed to operate in parallel with the training process. Extensive experiments on two real-world multimodal datasets show that the proposed QQR algorithm consistently outperforms benchmarks under modality imbalance conditions with promising learning performance.

多模态学习联邦学习边缘计算数据不平衡

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。