将多线性奇异值分解扩展为可核化的对偶形式,支持非线性建模。
A Kernelizable Primal-Dual Formulation of the Multilinear Singular Value Decomposition
- 提出多线性奇异值分解的对偶优化形式,兼具原始与对偶优势。
- 通过特征映射引入核方法,生成核张量实现非线性扩展。
- 适用于信号分析与深度学习中的高维数据建模,提升灵活性。
将学习任务表述为原问题与对偶问题的结合,是众多机器学习方法的核心。例如支持向量机(SVM)、最小二乘支持向量机(LS-SVM)、岭回归(RR)、Lasso回归(LR)、主成分分析(PCA)以及近期的奇异值分解(SVD)均可通过原权重或对偶拉格朗日乘子定义。原形式在样本量大时计算更优,对偶形式则适合高维数据。关键在于,通过在原问题中引入特征映射,可实现非线性化,对应于对偶中的核技巧。本文推导了多线性奇异值分解(MLSVD)的原-对偶形式,其特殊情形包括PCA与SVD。该形式不仅带来计算优势,还通过特征映射提出非线性扩展,导致对偶问题中出现核张量。讨论了其在信号分析与深度学习中的潜在应用。
原文摘要 · Abstract (English)
The ability to express a learning task in terms of a primal and a dual optimization problem lies at the core of a plethora of machine learning methods. For example, Support Vector Machine (SVM), Least-Squares Support Vector Machine (LS-SVM), Ridge Regression (RR), Lasso Regression (LR), Principal Component Analysis (PCA), and more recently Singular Value Decomposition (SVD) have all been defined either in terms of primal weights or in terms of dual Lagrange multipliers. The primal formulation is computationally advantageous in the case of large sample size while the dual is preferred for high-dimensional data. Crucially, said learning problems can be made nonlinear through the introduction of a feature map in the primal problem, which corresponds to applying the kernel trick in the dual. In this paper we derive a primal-dual formulation of the Multilinear Singular Value Decomposition (MLSVD), which recovers as special cases both PCA and SVD. Besides enabling computational gains through the derived primal formulation, we propose a nonlinear extension of the MLSVD using feature maps, which results in a dual problem where a kernel tensor arises. We discuss potential applications in the context of signal analysis and deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。