用图神经网络提升多模态数据缺失时的补全效果
GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning
- 构建图结构融合所有可用模态信息进行补全
- 在基准数据集上达到当前最佳性能
- 适合医疗等真实场景中模态缺失的分布式学习
当前分布式多模态学习通常假设各客户端可获取全部模态信息，但现实中往往存在模态缺失。本文研究拼贴学习（patchwork learning），即不同客户端可用模态各异，目标是无监督地补全各客户端缺失模态。现有方法未能充分利用所有可观测模态，仅依赖部分模态。为此，我们提出GraphPL，结合图神经网络与拼贴学习，灵活整合所有可观测模态，并对噪声输入保持鲁棒性。实验表明，GraphPL在基准数据集上表现领先。在真实世界分布式电子健康记录数据集上的结果进一步显示，该方法能学习强下游特征，支持疾病预测等任务。
原文摘要 · Abstract (English)
Current research on distributed multi-modal learning typically assumes that clients can access complete information across all modalities, which may not hold in practice. In this paper, we explore patchwork learning, in which the modalities available to different clients vary, and the objective is to impute the missing modalities for each client in an unsupervised manner. Existing methods are shown not to fully utilize the modality information as they tend to rely on only a subset of the observed modalities. To address this issue, we propose GraphPL, which combines graph neural networks with patchwork learning to flexibly integrate all observed modalities and remains robust with noisy inputs. Experimental results show that GraphPL achieves SOTA performance on benchmark datasets. Our results on real-world distributed electronic health record dataset show GraphPL learns strong downstream features and enables tasks like disease prediction via superior modality imputation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。