纠正深度核学习中关于复杂度主导超参数的误解，指出重参数化引入新数据拟合项。
Response to Promises and Pitfalls of Deep Kernel Learning
- 通过重参数化揭示隐藏的数据拟合项，打破复杂度主导的假象。
- 证明数据拟合与复杂度仍共同决定核超参数，避免数据过相关。
- 适合关注高斯过程、核方法理论严谨性的研究者阅读。
本文回应《深度核学习的承诺与陷阱》（Ober 等，2021）一文。高斯过程的边际似然可分解为数据拟合项与复杂度惩罚项。Ober 等指出，若核函数可乘以信号方差系数，则通过对该参数进行重参数化并代入其最优值，可使重参数后的数据拟合项固定。他们据此认为复杂度惩罚项（核矩阵的对数行列式）将主导其他核超参数的确定，导致数据过相关。然而，我们证明该重参数化实际上引入了另一个数据拟合项，影响所有其他核超参数。因此，数据拟合与复杂度之间的平衡在超参数选择中依然起关键作用。
原文摘要 · Abstract (English)
This note responds to "Promises and Pitfalls of Deep Kernel Learning" (Ober et al., 2021). The marginal likelihood of a Gaussian process can be compartmentalized into a data fit term and a complexity penalty. Ober et al. (2021) shows that if a kernel can be multiplied by a signal variance coefficient, then reparametrizing and substituting in the maximized value of this parameter sets a reparametrized data fit term to a fixed value. They use this finding to argue that the complexity penalty, a log determinant of the kernel matrix, then dominates in determining the other values of kernel hyperparameters, which can lead to data overcorrelation. By contrast, we show that the reparametrization in fact introduces another data-fit term which influences all other kernel hyperparameters. Thus, a balance between data fit and complexity still plays a significant role in determining kernel hyperparameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。