arXiv:2512.22102cs.LG2025-12被引 1

用信息分解方法解耦多模态回归中各模态的贡献,提升可解释性。

Explainable Multimodal Regression via Information Decomposition

  • 基于部分信息分解,将模态表示拆分为独有、冗余和协同成分。
  • 在六个真实数据集上,预测精度和可解释性均优于现有方法。
  • 适合需要理解多模态融合机制的研究者,如脑年龄预测场景。

多模态回归旨在从异构输入源预测连续目标,通常依赖早期或晚期融合策略。然而,现有方法缺乏对各模态及其交互作用贡献的解耦与量化工具,限制了融合过程的可解释性。本文提出一种基于部分信息分解(PID)的新型多模态回归框架,将模态特异性表示分解为独有、冗余和协同成分。原始PID框架本质欠定,为此我们通过强制潜变量与变换后响应变量(经逆正态变换)的联合分布满足高斯性,实现PID项的解析计算。此外,推导出闭式条件独立正则化项,促进各模态内独有信息的分离。在六个真实世界数据集上的实验,包括大规模神经影像数据的脑年龄预测案例研究,表明本框架在预测精度和可解释性上均超越现有最优方法,同时支持高效推理中的模态选择。代码已开源:https://github.com/zhaozhaoma/PIDReg。

原文摘要 · Abstract (English)

Multimodal regression aims to predict a continuous target from heterogeneous input sources and typically relies on fusion strategies such as early or late fusion. However, existing methods lack principled tools to disentangle and quantify the individual contributions of each modality and their interactions, limiting the interpretability of multimodal fusion. We propose a novel multimodal regression framework grounded in Partial Information Decomposition (PID), which decomposes modality-specific representations into unique, redundant, and synergistic components. The basic PID framework is inherently underdetermined. To resolve this, we introduce inductive bias by enforcing Gaussianity in the joint distribution of latent representations and the transformed response variable (after inverse normal transformation), thereby enabling analytical computation of the PID terms. Additionally, we derive a closed-form conditional independence regularizer to promote the isolation of unique information within each modality. Experiments on six real-world datasets, including a case study on large-scale brain age prediction from multimodal neuroimaging data, demonstrate that our framework outperforms state-of-the-art methods in both predictive accuracy and interpretability, while also enabling informed modality selection for efficient inference. Implementation is available at https://github.com/zhaozhaoma/PIDReg.

多模态信息分解可解释性回归

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。