揭示奇异线性模型中贝叶斯信息准则失效原因，提出修正方法
Evidence Slopes and Effective Dimension in Singular Linear Models
- 用真实对数规范阈值（RLCT）替代参数数量作为有效维度
- 发现拉普拉斯近似误差随 log n 线性增长，与 (d/2 - λ) 相关
- 证据斜率可作有效维度估计，对过完备重参数化不变
贝叶斯模型选择常依赖拉普拉斯近似或贝叶斯信息准则（BIC），假设有效模型维度等于参数数量。奇异学习理论以真实对数规范阈值（RLCT）取代该假设，其值在过参数化或秩亏模型中可能严格小于参数数量。本文研究线性高斯秩模型与线性子空间（字典）模型，其中边缘似然有闭式解且RLCT可解析计算。理论与实验表明，拉普拉斯/BIC的误差随 (d/2 - λ)·log n 线性增长，其中 d 为环境参数维度，λ 为 RLCT。基于 RLCT 的修正项可恢复正确的证据斜率，且对表示同一数据子空间的过完备重参数化保持不变。结果为奇异模型中拉普拉斯近似失败提供了明确的有限样本表征，并证明证据斜率可在简单线性设置中作为有效维度的实用估计器。
原文摘要 · Abstract (English)
Bayesian model selection commonly relies on Laplace approximation or the Bayesian Information Criterion (BIC), which assume that the effective model dimension equals the number of parameters. Singular learning theory replaces this assumption with the real log canonical threshold (RLCT), an effective dimension that can be strictly smaller in overparameterized or rank-deficient models. We study linear-Gaussian rank models and linear subspace (dictionary) models in which the exact marginal likelihood is available in closed form and the RLCT is analytically tractable. In this setting, we show theoretically and empirically that the error of Laplace/BIC grows linearly with (d/2 minus lambda) times log n, where d is the ambient parameter dimension and lambda is the RLCT. An RLCT-aware correction recovers the correct evidence slope and is invariant to overcomplete reparameterizations that represent the same data subspace. Our results provide a concrete finite-sample characterization of Laplace failure in singular models and demonstrate that evidence slopes can be used as a practical estimator of effective dimension in simple linear settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。