揭示了奇异模型中WAIC与WBIC的渐近关联,提升模型选择效率。
An Asymptotic Equation Linking WAIC and WBIC in Singular Models
- 通过奇异学习理论推导出WAIC与WBIC的渐近关系式。
- 用WBIC后验可无偏估计WAIC,避免重复采样。
- 为奇异模型高效模型选择提供理论基础,适合统计学习研究者。
在统计学习中,模型分为正则与奇异两类,取决于参数到概率分布的映射是否单射。大多数具有层次结构或隐变量的模型属于奇异模型,传统准则如AIC和BIC因似然和后验的正态近似失效而无法适用。为此提出了广义信息准则WAIC和广义贝叶斯信息准则WBIC。由于二者依赖于不同温度设置下的后验分布,通常需分别进行后验采样。本文从理论上推导出WAIC与WBIC之间的渐近方程,尽管二者基于不同的后验。该方程给出了以WBIC所用后验表示的WAIC的渐近无偏表达式。结果阐明了这两类准则在奇异学习理论框架内的结构关系,并深化了对其渐近行为的理解。这一理论贡献为未来奇异模型中模型选择的计算效率提升奠定了基础。
原文摘要 · Abstract (English)
In statistical learning, models are classified as regular or singular depending on whether the mapping from parameters to probability distributions is injective. Most models with hierarchical structures or latent variables are singular, for which conventional criteria such as the Akaike Information Criterion and the Bayesian Information Criterion are inapplicable due to the breakdown of normal approximations for the likelihood and posterior. To address this, the Widely Applicable Information Criterion (WAIC) and the Widely Applicable Bayesian Information Criterion (WBIC) have been proposed. Since WAIC and WBIC are computed using posterior distributions at different temperature settings, separate posterior sampling is generally required. In this paper, we theoretically derive an asymptotic equation that links WAIC and WBIC, despite their dependence on different posteriors. This equation yields an asymptotically unbiased expression of WAIC in terms of the posterior distribution used for WBIC. The result clarifies the structural relationship between these criteria within the framework of singular learning theory, and deepens understanding of their asymptotic behavior. This theoretical contribution provides a foundation for future developments in the computational efficiency of model selection in singular models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。