arXiv:2410.17700cs.LGcs.AI2024-10TPAMI

提出可扩展的随机特征潜变量模型,解决大规模数据下的计算瓶颈。

Scalable Random Feature Latent Variable Models

论文配图:Scalable Random Feature Latent Variable Models
图 1 · 摘自论文原文
  • 用块坐标下降变分推断替代蒙特卡洛采样,提升效率
  • 引入迪利克雷过程显式概率密度,支持高效优化
  • 适用于海量数据,擅长生成潜空间表示与缺失值补全

随机特征潜变量模型(RFLVMs)是当前最先进的潜变量模型,能处理非高斯似然并有效挖掘高维数据中的模式。然而,其严重依赖蒙特卡洛采样导致可扩展性差,难以应用于大规模观测数据集。为此,本文采用基于优化的变分贝叶斯推断(VBI)以克服采样方法的局限性。但现有VBI算法与RFLVM不兼容,且核学习中迪利克雷过程(DP)缺乏显式概率密度函数。针对这些问题,我们提出一种迪利克雷过程的棒棒糖构造法以获得显式PDF,并设计新型变分推断算法——块坐标下降变分推断(BCD-VI)。由此构建的可扩展随机特征潜变量模型(SRFLVM)在多个真实数据集上展现出良好可扩展性、计算效率,生成更具信息量的潜空间表示,并具备优异的缺失数据补全能力,优于当前主流方法。

原文摘要 · Abstract (English)

Random feature latent variable models (RFLVMs) represent the state-of-the-art in latent variable models, capable of handling non-Gaussian likelihoods and effectively uncovering patterns in high-dimensional data. However, their heavy reliance on Monte Carlo sampling results in scalability issues which makes it difficult to use these models for datasets with a massive number of observations. To scale up RFLVMs, we turn to the optimization-based variational Bayesian inference (VBI) algorithm which is known for its scalability compared to sampling-based methods. However, implementing VBI for RFLVMs poses challenges, such as the lack of explicit probability distribution functions (PDFs) for the Dirichlet process (DP) in the kernel learning component, and the incompatibility of existing VBI algorithms with RFLVMs. To address these issues, we introduce a stick-breaking construction for DP to obtain an explicit PDF and a novel VBI algorithm called ``block coordinate descent variational inference" (BCD-VI). This enables the development of a scalable version of RFLVMs, or in short, SRFLVM. Our proposed method shows scalability, computational efficiency, superior performance in generating informative latent representations and the ability of imputing missing data across various real-world datasets, outperforming state-of-the-art competitors.

潜变量模型变分推断可扩展性缺失数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。