提出高效计算多模态信息分解的新方法,适用于高维连续数据。
Partial Information Decomposition via Normalizing Flows in Latent Gaussian Distributions
- 基于高斯分布重构优化问题,提升计算效率
- 在合成数据上准确率优于现有方法,计算速度显著更快
- 适合处理真实世界多模态数据,如医疗或传感器融合
多模态分析在预测建模、数据融合和可解释性方面受到广泛关注。部分信息分解(PID)作为信息论框架,可量化多个模态对目标变量的独立、冗余或协同贡献。然而,现有方法依赖于基于估计的成对分布构建联合分布,对连续高维数据代价高且不准确。本文首次发现:当成对分布为多元高斯时,该问题可高效求解,称为高斯PID(GPID)。我们提出一种基于梯度的新算法,通过重构优化问题大幅提高计算效率。为进一步拓展至非高斯数据,引入信息保持编码器,将任意输入分布映射为成对高斯分布。同时解决了关于高斯联合解最优性的开放问题。在多种合成数据上验证表明,本方法比基线更准确高效;在大规模多模态基准测试中也展现出实际应用价值,可用于评估模型性能与选择最优模型。
原文摘要 · Abstract (English)
The study of multimodality has garnered significant interest in fields where the analysis of interactions among multiple information sources can enhance predictive modeling, data fusion, and interpretability. Partial information decomposition (PID) has emerged as a useful information-theoretic framework to quantify the degree to which individual modalities independently, redundantly, or synergistically convey information about a target variable. However, existing PID methods depend on optimizing over a joint distribution constrained by estimated pairwise probability distributions, which are costly and inaccurate for continuous and high-dimensional modalities. Our first key insight is that the problem can be solved efficiently when the pairwise distributions are multivariate Gaussians, and we refer to this problem as Gaussian PID (GPID). We propose a new gradient-based algorithm that substantially improves the computational efficiency of GPID based on an alternative formulation of the underlying optimization problem. To generalize the applicability to non-Gaussian data, we learn information-preserving encoders to transform random variables of arbitrary input distributions into pairwise Gaussian random variables. Along the way, we resolved an open problem regarding the optimality of joint Gaussian solutions for GPID. Empirical validation in diverse synthetic examples demonstrates that our proposed method provides more accurate and efficient PID estimates than existing baselines. We further evaluate a series of large-scale multimodal benchmarks to show its utility in real-world applications of quantifying PID in multimodal datasets and selecting high-performing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。