提出拉普拉斯核及其推广的高效随机特征映射方法
Feature maps for the Laplacian kernel and its generalizations
- 设计带重尾弱耦合随机权重的采样方案
- 在真实数据集上验证了特征映射的逼近效果
- 适用于需要稳定核方法的深度学习场景
近年来，由于拉普拉斯核对带宽超参数更稳定且表达能力等同于深层全连接网络的神经正切核，其在机器学习中的应用重新受到关注。然而，与高斯核不同，拉普拉斯核不可分离，给近似方法（尤其是随机傅里叶特征RFF）带来挑战。本文为拉普拉斯核及其两种推广形式——Matérn核和指数幂核，提供了随机特征映射。我们提出了可高效实现的权重矩阵采样方案，使随机特征能有效逼近这些核函数。通过在真实数据集上的数值实验，验证了该方法的有效性。
原文摘要 · Abstract (English)
Recent applications of kernel methods in machine learning have seen a renewed interest in the Laplacian kernel, due to its stability to the bandwidth hyperparameter in comparison to the Gaussian kernel, as well as its expressivity being equivalent to that of the neural tangent kernel of deep fully connected networks. However, unlike the Gaussian kernel, the Laplacian kernel is not separable. This poses challenges for techniques to approximate it, especially via the random Fourier features (RFF) methodology and its variants. In this work, we provide random features for the Laplacian kernel and its two generalizations: Matérn kernel and the Exponential power kernel. We provide efficiently implementable schemes to sample weight matrices so that random features approximate these kernels. These weight matrices have a weakly coupled heavy-tailed randomness. Via numerical experiments on real datasets we demonstrate the efficacy of these random feature maps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。