构建可调控模态互信息的多模态数据集，用于系统评估自监督学习方法。
Multimodal Datasets with Controllable Mutual Information
- 用流模型与因果框架生成有已知互信息的多模态数据。
- 验证模态间互信息越高，回归性能越好。
- 适用于天体物理多探测器和高复杂度自监督学习研究。
我们提出一种生成具有明确互信息（MI）的多模态数据集的框架，支持对互信息估计器和多模态自监督学习（SSL）方法的系统性研究。该框架基于流模型和结构化因果框架，生成具有真实感且已知互信息的潜在变量。我们在不同真实互信息值的数据集上测试了一系列互信息估计器，并验证了输入模态与目标值之间的互信息越高，回归性能越优。最后，阐述了该框架在多探测器天体物理及高度多模态自监督学习场景中的应用潜力。
原文摘要 · Abstract (English)
We introduce a framework for generating highly multimodal datasets with explicitly calculable mutual information (MI) between modalities. This enables the construction of benchmark datasets that provide a novel testbed for systematic studies of mutual information estimators and multimodal self-supervised learning (SSL) techniques. Our framework constructs realistic datasets with known MI using a flow-based generative model and a structured causal framework for generating correlated latent variables. We benchmark a suite of MI estimators on datasets with varying ground truth MI values and verify that regression performance improves as the MI increases between input modalities and the target value. Finally, we describe how our framework can be applied to contexts including multi-detector astrophysics and SSL studies in the highly multimodal regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。