解决眼科多模态数据缺失问题,提升疾病分级诊断准确率。
Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis
- 通过互信息引导解耦特征,分离出模态共性和特异性信息。
- 在四个眼科数据集上显著优于当前最佳方法,提升诊断性能。
- 适合处理真实医疗场景中缺失模态的多模态学习任务。
眼科医生通常依赖多模态数据以提高临床决策的准确性。然而,在实际场景中,由于设备短缺、数据质量差及隐私问题,多模态数据缺失普遍存在。现有深度学习方法通常通过学习不同模态组合的隐式潜在子空间来应对,但存在两大局限:(1) 隐式表示限制了模型捕捉模态特异性信息的能力;(2) 模态异质性导致特征分布差异和冗余。为此,我们提出不完整模态解耦表示(IMDR)策略,通过互信息引导,将特征显式解耦为模态共性和模态特异性部分,从而重建有价值的缺失语义并生成鲁棒的多模态表示。此外,引入联合代理学习模块,利用各类别提取的代理信息消除模态内冗余。在四个眼科多模态数据集上的实验表明,所提IMDR显著优于当前最优方法。
原文摘要 · Abstract (English)
Ophthalmologists typically require multimodal data sources to improve diagnostic accuracy in clinical decisions. However, due to medical device shortages, low-quality data and data privacy concerns, missing data modalities are common in real-world scenarios. Existing deep learning methods tend to address it by learning an implicit latent subspace representation for different modality combinations. We identify two significant limitations of these methods: (1) implicit representation constraints that hinder the model's ability to capture modality-specific information and (2) modality heterogeneity, causing distribution gaps and redundancy in feature representations. To address these, we propose an Incomplete Modality Disentangled Representation (IMDR) strategy, which disentangles features into explicit independent modal-common and modal-specific features by guidance of mutual information, distilling informative knowledge and enabling it to reconstruct valuable missing semantics and produce robust multimodal representations. Furthermore, we introduce a joint proxy learning module that assists IMDR in eliminating intra-modality redundancy by exploiting the extracted proxies from each class. Experiments on four ophthalmology multimodal datasets demonstrate that the proposed IMDR outperforms the state-of-the-art methods significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。