首次精确计算二维奇异模型的学习系数，为贝叶斯模型选择提供理论基础。
Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models
- 基于代数几何方法，首次实现二维模型中学习系数的精确求解。
- 算法适用于KL散度等价于多项式的模型，可处理多项式神经网络等实例。
- 相比采样方法更快速且揭示隐藏代数结构，适合理论研究者使用。
经典信息准则（如BIC）依赖于正则性假设，在奇异模型中失效，导致深度学习等场景下模型选择错误。广义贝叶斯信息准则（WBIC）依赖局部学习系数λ，其在解析情况下与模型的KL散度的实对数规范阈值（RLCT）一致，以捕捉正确的边缘似然渐近行为。以往学习系数的精确计算仅限于特例，普遍依赖采样估计。本文提出首个确定性算法，可对任意二维模型（其KL距离与多项式接触等价）精确计算局部RLCT，给出复杂度上界，并在多项式神经网络等广泛模型类中验证有效性。该结果不仅为校准采样估计提供真值，还揭示了采样无法发现的代数结构，且在浅层情形下速度优于采样方法。
原文摘要 · Abstract (English)
Classical information criteria such as the Bayesian Information Criterion (BIC) rely on regularity assumptions that break down for singular models, leading to incorrect model selection in settings such as deep learning. The Widely Applicable Bayesian Information Criterion (WBIC) relies on local learning coefficients $λ$, which in the analytic case coincides with local Real Log Canonical Thresholds (RLCT) of the Kullback-Leibler divergence of the model, to capture correct marginal likelihood asymptotics. Exact computation of the learning coefficients has been limited to special cases, and only sampling-based estimation methods are generally applicable. We present the first deterministic algorithm that computes local RLCTs exactly for any two-dimensional model whose Kullback-Leibler distance is contact equivalent to a polynomial, derive a bound on its complexity, and demonstrate its effectiveness for a broad class of models, with applications including polynomial neural networks. Beyond providing ground truth to calibrate sampling-based estimators, exact computation reveals algebraic structure in learning coefficients that sampling cannot and out-speeds it in the shallow regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。