深度高斯过程在多项式核下易因超参不当导致先验坍缩。
Support Collapse of Deep Gaussian Processes with Polynomial Kernels for a Wide Regime of Hyperparameters
- 利用贝里-埃斯塞定理分析深层高斯过程的先验分布
- 浅层即出现平均效应，深度增加时先验快速坍缩至零
- 揭示超参敏感性，指导模型设计与调参
我们分析了使用多项式核的深度高斯过程所诱导的先验分布。即使在相对较小的深度下，该深度高斯过程内部也出现平均效应，其先验可有效通过贝里-埃斯塞定理进行分析与近似。关键发现之一是：若未进行精细的超参数调优，深度高斯过程的先验会随着深度增加而迅速坍缩至零，或对低范数函数几乎不赋予概率质量。这一现象与实验观察高度一致，并与基于卷积的深度高斯过程已知结果相吻合。
原文摘要 · Abstract (English)
We analyze the prior that a Deep Gaussian Process with polynomial kernels induces. We observe that, even for relatively small depths, averaging effects occur within such a Deep Gaussian Process and that the prior can be analyzed and approximated effectively by means of the Berry-Esseen Theorem. One of the key findings of this analysis is that, in the absence of careful hyper-parameter tuning, the prior of a Deep Gaussian Process either collapses rapidly towards zero as the depth increases or places negligible mass on low norm functions. This aligns well with experimental findings and mirrors known results for convolution based Deep Gaussian Processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。