提出深层Q指数过程，提升图像等复杂数据的建模能力
Deep Q-Exponential Processes
- 用分层Q指数过程替代传统高斯过程，实现更灵活的正则化
- 通过诱导点稀疏近似和变分推断，支持大规模数据高效训练
- 在图像等非均匀数据上表现优于现有深度概率模型
受深度神经网络启发，深度高斯过程（DGP）通过堆叠多层高斯过程提升表达能力。然而，作为$L_2$正则先验的高斯过程对边缘等非均匀结构过度平滑，效果不佳。近期提出的Q指数过程（Q-EP）是高斯过程的$L_q$松弛，通过参数$q>0$调节正则化特性，当$q=2$时即为标准高斯过程。与高斯过程一样，Q-EP具备可追踪的后验与预测分布，也可堆叠以增强建模灵活性。本文将Q-EP推广至深层Q-EP，结合浅层Q-EP作为潜在变量模型，并构建其层级结构，同时引入基于诱导点的稀疏近似与可扩展变分策略以实现高效推断。实验表明，所提深层Q-EP在多个前沿深度概率模型对比中展现出显著数值优势。
原文摘要 · Abstract (English)
Motivated by deep neural networks, the deep Gaussian process (DGP) generalizes the standard GP by stacking multiple layers of GPs. Despite the enhanced expressiveness, GP, as an $L_2$ regularization prior, tends to be over-smooth and sub-optimal for inhomogeneous subjects, such as images with edges. Recently, Q-exponential process (Q-EP) has been proposed as an $L_q$ relaxation to GP and demonstrated with more desirable regularization properties through a parameter $q>0$ with $q=2$ corresponding to GP. Sharing the similar tractability of posterior and predictive distributions with GP, Q-EP can also be stacked to improve its modeling flexibility. In this paper, we generalize Q-EP to deep Q-EP to enjoy both proper regularization and improved expressiveness. The generalization is realized by introducing shallow Q-EP as a latent variable model and then building a hierarchy of the shallow Q-EP layers. Sparse approximation by inducing points and scalable variational strategy are applied to facilitate the inference. We demonstrate the numerical advantages of the proposed deep Q-EP model by comparing with multiple state-of-the-art deep probabilistic models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。