比较KL散度与Wasserstein距离在实验设计中的优劣,指导实际应用选择。
Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
- 用玩具模型揭示Wasserstein距离受后验位置影响,可能产生虚假信息奖励
- 在无模型偏差时KL散度收敛更快,在有偏差时Wasserstein更稳健
- 为复杂系统实验设计提供实用的效用函数选择依据
设计能系统采集复杂物理系统数据的实验,对加速科学发现至关重要。贝叶斯实验设计(BED)提供了一个融合实验规划与概率推断的信息驱动框架,但效用函数的选择长期存在争议。尽管KL散度是最常用选项,近期研究提出使用Wasserstein距离作为替代。本文首先通过一个简单例子说明:固定形状的后验分布,其Wasserstein距离值会因主质量在支撑集中的相对位置而变化,尤其在非信息性先验(如均匀分布)下,可能出现与信息增益无关的虚假奖励。随后,我们通过经典源反演问题系统对比两种准则,发现当无模型偏差时KL散度收敛更快;若模型偏差不可忽略,Wasserstein度量则提供更鲁棒的序贯实验设计结果。这些发现明确了两者在效用函数中的权衡,为实际应用提供了选择指南。
原文摘要 · Abstract (English)
Designing experiments that systematically gather data from complex physical systems is central to accelerating scientific discovery. While Bayesian experimental design (BED) provides a principled, information-based framework that integrates experimental planning with probabilistic inference, the selection of utility functions in BED is a long-standing and active topic, where different criteria emphasize different notions of information. Although Kullback--Leibler (KL) divergence has been one of the most common choices, recent studies have proposed Wasserstein distance as an alternative. In this work, we first employ a toy example to illustrate an issue of Wasserstein distance - the value of Wasserstein distance of a fixed-shape posterior depends on the relative position of its main mass within the support and can exhibit false rewards unrelated to information gain, especially with a non-informative prior (e.g., uniform distribution). We then further provide a systematic comparison between these two criteria through a classical source inversion problem in the BED literature, revealing that the KL divergence tends to lead to faster convergence in the absence of model discrepancy, while Wasserstein metrics provide more robust sequential BED results if model discrepancy is non-negligible. These findings clarify the trade-offs between KL divergence and Wasserstein metrics for the utility function and provide guidelines for selecting suitable criteria in practical BED applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。