挖掘医学多模态数据中必要且充分的特征，提升模型性能与鲁棒性。
Seeking Necessary and Sufficient Information from Multimodal Medical Data
- 分离模态共享与特有特征，构建可计算的必要充分性目标
- 在真实医疗数据上验证，模型对缺失模态更鲁棒且准确率提升
- 适合需要高可靠性医疗决策系统的研究者使用
从医学影像与其他数据源中学习多模态表示可为决策提供更丰富信息。尽管已有多种多模态模型，但均未关注同时具备必要性（结果发生所必需）和充分性（足以决定结果）的特征。我们主张学习此类特征至关重要：它们能捕获关键预测信息以提升模型性能，并增强对模态缺失的鲁棒性，因每种模态均能提供足够的预测信号。通过引入概率必要性与充分性（PNS）作为学习目标，该方法在单模态场景中已被证明有效。然而，将PNS扩展至多模态场景仍具挑战，因关键假设被违反。为此，我们提出将多模态表示分解为模态不变与模态特定成分，并分别为其推导出可操作的PNS目标。在合成及真实世界医学数据集上的实验表明该方法有效。代码将公开于GitHub。
原文摘要 · Abstract (English)
Learning multimodal representations from medical images and other data sources can provide richer information for decision-making. While various multimodal models have been developed for this, they overlook learning features that are both necessary (must be present for the outcome to occur) and sufficient (enough to determine the outcome). We argue learning such features is crucial as they can improve model performance by capturing essential predictive information, and enhance model robustness to missing modalities as each modality can provide adequate predictive signals. Such features can be learned by leveraging the Probability of Necessity and Sufficiency (PNS) as a learning objective, an approach that has proven effective in unimodal settings. However, extending PNS to multimodal scenarios remains underexplored and is non-trivial as key conditions of PNS estimation are violated. We address this by decomposing multimodal representations into modality-invariant and modality-specific components, then deriving tractable PNS objectives for each. Experiments on synthetic and real-world medical datasets demonstrate our method's effectiveness. Code will be available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。