提出各向同性核的谱混合表示,简化了多种核函数的随机傅里叶特征采样。
A spectral mixture representation of isotropic kernels with application to random Fourier features
- 将各向同性核的谱分布表示为α-稳定向量的尺度混合
- 给出指数幂、广义柯西等核的显式采样公式
- 适用于支持向量机、高斯过程等核方法,提升计算效率
Rahimi与Recht(2007)提出通过从核的谱分布中随机采样来分解正定平移不变核,该技术称为随机傅里叶特征(RFF),理论上可应用于所有谱分布可识别并模拟的核。然而实践中通常仅用于高斯核,因其谱分布也为高斯型,计算简便。本文证明:在任意维度d≥1的欧氏空间中,正定各向同性核的谱分布可表示为α-稳定随机向量的尺度混合,并明确给出了混合分布与核函数的关系。这一构造性分解为多变量正定平移不变核提供了简单且直接可用的谱采样公式,涵盖指数幂核、广义柯西核,以及新提出的广义马特恩、三克米和福克斯H核。特别地,我们推导出这些核的谱分布(仅能以福克斯H函数显式表达)实际上是多维高斯分布的尺度混合,且给出了明确的混合分布公式。该结果广泛适用于支持向量机、核岭回归、高斯过程等依赖随机傅里叶特征的核方法。
原文摘要 · Abstract (English)
Rahimi and Recht (2007) introduced the idea of decomposing positive definite shift-invariant kernels by randomly sampling from their spectral distribution for machine learning applications. This famous technique, known as Random Fourier Features (RFF), is in principle applicable to any such kernel whose spectral distribution can be identified and simulated. In practice, however, it is usually applied to the Gaussian kernel because of its simplicity, since its spectral distribution is also Gaussian. Clearly, simple spectral sampling formulas would be desirable for broader classes of kernels. In this paper, we show that the spectral distribution of positive definite isotropic kernels in $\mathbb{R}^{d}$ for all $d\geq1$ can be decomposed as a scale mixture of $α$-stable random vectors, and we identify the mixing distribution as a function of the kernel. This constructive decomposition provides a simple and ready-to-use spectral sampling formula for many multivariate positive definite shift-invariant kernels, including exponential power kernels, and generalized Cauchy kernels, as well as newly introduced kernels such as the generalized Matérn, Tricomi, and Fox $H$ kernels. In particular, we retrieve the fact that the spectral distributions of these kernels, which can only be explicited in terms of the Fox $H$ special function, are scale mixtures of the multivariate Gaussian distribution, along with an explicit mixing distribution formula. This result has broad applications for support vector machines, kernel ridge regression, Gaussian processes, and other kernel-based machine learning techniques for which the random Fourier features technique is applicable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。