为多模态回归主动学习设计了能捕捉认知不确定性的新方法
A Mutual Information Lower Bound for Multimodal Regression Active Learning

- 用双索引框架分离认知与随机不确定性
- 提出可计算的互信息下界,显著提升多模态预测性能
- 适合处理输出分布多峰的回归任务,尤其在复杂场景中表现稳定
连续回归的主动学习缺乏针对多模态预测分布中认知不确定性的采集函数:方差无法捕捉模式间的分歧,而基于信息论的目标(如BALD)专为离散输出设计。本文提出双索引框架,显式分离认知不确定性(由随机索引选择模型假设)与随机不确定性(由第二索引控制假设内随机性)。该框架内的熵分解识别出输出与认知索引之间的互信息作为合理采集目标,并证明该量随训练数据增长趋于零,确认其准确捕获了数据可解决的不确定性。由于该互信息对连续输出不可计算,我们推导出互信息下界(MI-LB)采集函数,为混合密度网络集成提供闭式近似。在包含多模态系统的基准测试中,MI-LB匹配或超越所有基线,且唯一实现一致领先;几何与费舍尔基线仅在输入空间已编码多模态时有效,否则崩溃。
原文摘要 · Abstract (English)
Active learning for continuous regression has lacked an acquisition function that targets epistemic uncertainty when the predictive distribution is multimodal: variance misses modal disagreement, and information-theoretic targets like BALD are designed for discrete outputs. We introduce a Two-Index framework that makes this separation explicit: one stochastic index selects among competing model hypotheses (epistemic source), while a second governs within-hypothesis randomness (aleatoric source). An entropy decomposition within the framework identifies the mutual information between the output and the epistemic index as a principled acquisition objective, and we prove this quantity vanishes as the model is trained on growing datasets, confirming that it captures exactly the uncertainty data can resolve. Because this mutual information is intractable for continuous outputs, we derive the Mutual Information Lower Bound (MI-LB) acquisition function, a closed-form approximation for Mixture Density Network ensembles. On benchmarks featuring multimodal systems, MI-LB matches or beats every baseline evaluated and is the only method to do so consistently -- geometric and Fisher-based baselines compete only when the input space already encodes the multimodality, and collapse otherwise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。