提出MoSARe模型，解决医疗多模态数据缺失问题。
Towards Robust Multimodal Representation: A Unified Approach with Adaptive Experts and Alignment
- 采用专家自适应选择与跨模态注意力融合特征
- 在完整数据下准确率优于现有模型，在缺损时仍保持稳定
- 适合资源有限的临床场景，尤其应对隐私或成本导致的数据缺失
医疗诊断依赖医学影像、基因信息和临床记录等多源数据，但因隐私限制、成本和技术问题，数据缺失普遍存在，导致现有多模态模型不可靠。为此，我们提出一种名为Mixture of Experts, Symmetric Aligning, and Reconstruction（MoSARe）的新框架，可在不完整多模态数据下保持高精度。该模型融合专家选择、跨模态注意力与对比学习机制，增强特征表示与决策能力。实验表明，当数据完整时，MoSARe性能优于现有方法；即使部分数据缺失，仍能提供可靠预测。其在真实医疗环境，特别是资源受限场景中具有显著应用价值。代码已开源：https://github.com/NazaninMn/MoSARe。
原文摘要 · Abstract (English)
Healthcare relies on multiple types of data, such as medical images, genetic information, and clinical records, to improve diagnosis and treatment. However, missing data is a common challenge due to privacy restrictions, cost, and technical issues, making many existing multi-modal models unreliable. To address this, we propose a new multi-model model called Mixture of Experts, Symmetric Aligning, and Reconstruction (MoSARe), a deep learning framework that handles incomplete multimodal data while maintaining high accuracy. MoSARe integrates expert selection, cross-modal attention, and contrastive learning to improve feature representation and decision-making. Our results show that MoSARe outperforms existing models in situations when the data is complete. Furthermore, it provides reliable predictions even when some data are missing. This makes it especially useful in real-world healthcare settings, including resource-limited environments. Our code is publicly available at https://github.com/NazaninMn/MoSARe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。