提出改进的多模态融合方法，提升生物医学分类效果。
Exploring Multi-Modality Dynamics: Insights and Challenges in Multimodal Fusion for Biomedical Tasks
- 结合特征与模态重要性动态融合多模态数据
- 特征重要性有效提升性能与可解释性
- 新方法适用于图像数据，适合医疗影像分析研究者
本文研究了Han等人（2022）提出的用于生物医学分类任务的多模态动态融合方法（MM dynamics）。该算法通过整合特征级和模态级的信息量，实现动态多模态融合以提升分类性能。然而，我们的分析发现，在复现和扩展该方法时存在若干局限与挑战：特征重要性有助于提升性能与可解释性，而模态重要性并未带来显著优势，甚至可能导致性能下降。基于此，我们将特征重要性扩展至图像数据，提出了Image MM dynamics。尽管该方法在定性上表现良好，但在定量评估中未超越基线方法。
原文摘要 · Abstract (English)
This paper investigates the MM dynamics approach proposed by Han et al. (2022) for multi-modal fusion in biomedical classification tasks. The MM dynamics algorithm integrates feature-level and modality-level informativeness to dynamically fuse modalities for improved classification performance. However, our analysis reveals several limitations and challenges in replicating and extending the results of MM dynamics. We found that feature informativeness improves performance and explainability, while modality informativeness does not provide significant advantages and can lead to performance degradation. Based on these results, we have extended feature informativeness to image data, resulting in the development of Image MM dynamics. Although this approach showed promising qualitative results, it did not outperform baseline methods quantitatively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。